You can spray aphids with regular very diluted dish soapy water or plain water... it will kill them to be knocked off. I assure you as things warm up the bugs(lots) that eat these lil buggers(aphids) will show up. They've done that routine with me for over a decade.
I'm not talking a few aphids either. I'm talking an entire rose one big,moving mass that was totally clean a month later. PJ

Thank you PJ. Will the soap mix kill the lady bugs too?

Yes, the soap mixture will probably hurt the ladybugs, especially their larvae which are voracious eaters. You can do what PJ said by hosing them off. The ladybugs and their larvae will find them within a few days. Once they find the aphids, they will eat all the aphids within a day or two. At least that is what happened when I had aphids on my coral honeysuckle. I released ladybugs and within two days the aphids were gone and the ladybugs had reproduced and little larvae were crawling all over eating any leftover or new aphids or any other soft bodied bug they found.

You know, I'm sitting here in a quiet house, and keep hearing things run into the window. *shudder* Who knows what is flying around out there even at this very moment?! Probably June bugs, but maybe also roaches, etc. I haven't had many aphids yet but I expect them soon, as they arrived en grande masse last year to eat my unhealthy spirea. The plants are happier this year because we cut them back, butI still expect the aphids, since they are in a bad location that doesn't get much airflow or sunlight. I didn't have much luck with the natural methods last year (water blasting, and dishsoap/boiled garlic/water) but I'll try all that stuff again this year if necessary. Actually I will probably use orange oil in some manner.

Personally, I'd move the spirea or plant another one. That's a lot of work for a spirea. PJ