Sounds like some type of caterpillar. What zone are you? Can you take a sharp picture of them. Hand pick them & drop into soapy water is one slow solution.

Zone 4. Little green worms are eating my azalea bush leaves. Red lily beetle are eating my asiatic lilys, I'm ready to give up. I have to be careful what I use because of a 5 lb. Yorkie and two labs.

For all types of caterpillars you can use Bt (bacillus thuringiensis), an organic control that only affects caterpillars when they eat your plants. It gives them a tummy ache that makes them stop feeding, and then they crystallize. It's very satisfying. Definitely will not hurt you or your dogs, nor other insects.