My gardenia often drops a few buds, with no sign of creepy crawlies.

I have 3 different varieties of passion vines and often the buds drop off. I never see any bugs, either, so it's something about the growing conditions. I get enough blooms anyhow, so the same may happen for you.
Thrips are tiny little bud eaters and you'll see them if you open up a bud - I see them on hibiscus - hard to get rid off so I hope that's not what you have.
Hester

On mine, the blossoms pop open mid-morning and fall off at day's end.

Hmm... thanks everyone. I guess I will just have to keep an eye on my vines and hopefully catch one in bloom. Thanks again.

IMHO, if you live in Florida, you have SPIDER MITES at this time of year. Maybe you can't see them, but they are there, and causing flower buds to abort. Spray the plants with mild, soapy water 2-3 times and see if that makes a difference.

That might explain why the baby Cuban Anoles are hopping among the trellis seemingly munching on things that I haven't seen yet. But aside from the one fallen bud (so far), my passiflora seem to be in good health. The Morning Glory on the other hand... something (slugs I think) went after those with a vengeance.

Oh, and do not worry. I have checked the credentials of the Anoles and they are here and working legally. One claimed to be 15th generation locally-born Cuban-American Anole with 1/4 Brown Anole and 1/8 Crested Anole in its family bloodline. Now the Mediterranean Geckos... they keep evading me, so I'm guessing there's something slimy going on with them.