It sounds like a pollination issue. Are there any male blossoms present? If not, check each morning for the presence of both female and male open blossoms. If you find both, detach the male blossom, remove it's petals, and use the remaining stamen to pollinate any open female blossoms.
You might want to do this regardless as there simply may be no insects doing the job for you.