From "wikipedia.org":

There are several types of honey that are known to be toxic to humans. The most common of these in the northern hemisphere, popularly known as Mad Honey, is produced from the flowers of Rhododendrons, Mountain Laurels and azaleas. The nectar of these plants may contain Grayanotoxin, a compound which is both psychoactive and poisonous to humans but harmless to bees. The effects of Mad Honey have been reported in Western literature as early as 401 BC (See Xenophon's description of the effects of toxic honey in the Anabasis [1]) The shape of the Azalea flower, however, makes access to nectar difficult for honeybees. And during the time at which Azaleas bloom, there are usually other flowers available which are more appealing to the honeybee. So lethal honey is rarely encountered.

And I should add, a handful of flowers in your garden shouldn't have any real effect. Honeybees will visit some 2 million flowers to produce just one pound of honey.

-Tim

I think I also read that if it's fully cured, it's not a problem. When it's still uncapped, apparently it's worse.

I'll say I read that. I don't know if it's true.