Are they green, hairless caterpillars? If so cabbage looper sounds right.

You can prevent them from keeping the white moths off them with row covers. These plants do not require insect pollination so a row cover is a viable option right from the get go through harvest.

You can also use bacteria called bt. You will find it under various names in the store. Completely safe for everything other than caterpillars.

If the number of plants you have is small you may wish to simply inspect your plants daily and remove the loopers by hand.

The floating row covers need to be in place before the wee white butterflies appear. Once the larva, the small green colored leaf eating wormys thingys are on the plants the the only effective, organic, controls are squishing them between your fingers or spraying or dusting Bacillus thurigiensis - Kurstaki on the plants.

I've had excellent results with spinosad.

Keep in mind that the people at the OMRI have reservations about whether Spinosad is an acceptable organic pesticide.