Well..... I guess you could keep cutting off the blossoms to keep the highly beneficial wasps away. If it's the Swallowtails you're after then I suppose that's what you'll have to do,

The flowers attract lots of beneficials, both pollinators and pest predators. The flowers also produce the seed which is a very tasty herb in its own right. In my garden it also self sows, so the next generation of plants is guaranteed.

From my observations (from cutting the flowers for cooking) I don't think removing the flowers encourages more leaves either. To do that you'd need to remove the entire blooming stem to the ground which would actually lose you leaves.