All kinds of sprays exist, and no it won't taint the cherries one bit. If you grow cherries you have to spray for insects and fungus, and net the tree for birds. Otherwise you will never taste the fruit. Especially in your area. I'll let the more expierenced growers tell you what spray to use, I'm still learning and testing many different types.

I am growing in containers if that changes the potential advice.

As container trees don't usually get too big, you can probably just check the entire tree. If there are only a few, I'd get rid of them by hand, rather than trying to spray something. That's what I did last year on my apple trees (the little worms sound similar), and there was significantly more ground to cover.

Here is a pic of the top of the tree where the damage is. I didn't see any additional worms but I didn't check the tiny emerging leaves.