Well, first a comment and then I will be more helpful. ANY ornamental plant that you have to spray pesticides every 4 weeks on to enjoy is just ridiculous. Time to choose a plant better suited to the area. You have better things to do with your gardening time and garden enjoyment.

OK, now to be more specifically helpful. If what you are seeing is indeed a caterpillar you can easily control them by applying Bt, (i)Bacillus thuringiensis (/i).

Bt will kill the black guys as well as the any other wanted caterpillars you may find in your yard. So be very careful with your application. Most garden centers carry this product.

Here is a link that might be useful: Info on Bt

I'm curious...what is (are) the problem(s) with your pyracantha that you are having to spray so often?

You have
Spicebush Swallowtail caterpillars - lucky you! They are beautiful.

Not sure about the other one you described.

I have these guys on mine. Any ideas? Are they worth keeping?

Yes they are. Much too beautiful to be destroyed. There are lots of people who would jump for joy if they found these on their plants.