You don't give your location but if you're in the Northern hemisphere lemon balm will be dying down now anyway. It won't 'stay the same' because it's a herbaceous perennial, not an evergreen shrub. You could just cut it to the ground right now and deprive the caterpillars of their food source.

I would still be inclined to cut it to the ground. It gets ratty with age anyway. Just a thought, we're definitely talking about lemon balm and not lemon verbena aren't we?

I live in California where many "annuals" are perennial. They still need a hard cutting-back at least once a year.... or whenever they get ratty,,, or have pests.

Best plan is to cut it back, but be prepared for the next generation. Use a small cultivator in the soil around your plants to unearth hiding caterpillars and pupae. That's where these guys over winter and simply hang out during the day.

You might also think about obtaining some Thuricide or other biological control for caterpillars. Thuricide is a strain of bacterium, Bacillus thuringiensis var. Kurstaki, affects only caterpillars and only if they are feeding on the plant you've sprayed. It's acceptable for organic gardeners and can be used up to harvest.