Yes, it is to soon for it to be producing but it will probably abort any female blossoms that do open for a couple more weeks. Often times a squash won't have both male and female blossoms open at the same time until the plant is about 5-6 weeks old. My yellow summer squash plants seeded May 7 currently have several female proto fruit but no male blossoms had opened as of yesterday so they are probably not getting pollinated unless I have a neighbor nearby with some open male squash blossoms that I don't know about.

should I pinch them off or just leave them?

Just leave them be. If they are female blooms with the little proto fruit behind them they will probably abort (stop growing larger) after a week or 2 and you can harvest the small fruit then. If you leave them on they will start to rot from the blossom end.