That's the caterpillar of one of the Skipper butterflies, probably a Silver Spotted Skipper. Legumes are a primary host plant.

I'll post a picture for your comparison.

Here is a link that might be useful: Silver Spotted Skipper caterpillar.

That does look like its it. Also, the butterflies in the area look like those too. I think I can just remove them as I see them.

Yes, that's all you'll need to do. I always toss garden caterpillars to the birds or even put them in the bird bath.

They're strange looking little things, aren't they...with that odd head capsule?