Sounds just like sneaky thrips, which hide in flowers. If you are collecting the flowers, you can soak them in a water bath with a few drops of dish soap. Thrips will come out of hiding and sink. Give the flowers another bath in clean water then air dry.

I grow flowers for cutting and am amazed at the number of thrips that come out of hiding when I bring them inside.

If I determine that the thrip population is high enough to cause damage to the plants, I use azadirachtin (neem), which serves as an anti-feedant and IGR. Not too sure about using it on flower heads used for medicinal purposes, though. It might affect the taste, the ability to dry properly, etc.

Try cold pressed neem oil, it works with every bug I've had to destroy. I get mine from neem tree farms. ;)