When I looked at the first picture, I thought they might be salt marsh moth caterpillars or yellow bears. But then looking at the second picture, I think I see a faint red line along the side of the cat - do they have this red line? Do they tend to follow each other in a row like ants? If so, they're probably io moth caterpillars. Io moths lay their eggs on most anything, they're probably the only moth in the silk moth group that qualifies as a generalist. They reportedly sting, although not to the degree that saddle back caterpillars do.
Congratulations if I'm right! They should turn green as they get into further instars - they make a pretty cat and a pretty moth - here's a male I raised -
{{gwi:453230}}
Sherry

I did see an adult IO in my yard not too long ago so that might be it.

What could I move them to? There is a horde of them, lined up in parellel and I don't think that one year old Baptisia is up to the challenge!

They're known to eat most anything, so, if I were you, I'd move some to a tree/bush with a good bit of healthy new growth on it.
Do you have a wild black cherry tree? Sassafras? Willow? Tulip poplar? Those are some popular host plants. They even eat wax myrtle.
Sherry