This picture has one of the little bugs on the underside of the leaf. I know it's hard to see.

I have more pictures, just ask.

Your first photo shows azalea leaf gall... common fungal disease in the early spring on azaleas. Leaves of infected plants develop distorted growth. If only a few plants are affected and on each plant just a few stems/leaves, pick and destroy (meaning garbage not compost) galls, it rarely does enough damage to require chemical control. If the infection was extensive, a fungicide might be helpful but must be sprayed before the new foliage buds begin to open during or immediately after bloom....in my zone my own are past that now.

Your second photo is showing damage from azalea lace bug.

Here is a link that might be useful: Azalea lace bug

Thank you for your help, maybe I can begin to really help this poor plant. Also I meant to say that the bugs have a black body and a red head (I had it backwards).
luckily for me it seems that only one out of four is really suffering however is like to treat all of them if it's fungal, just to be safe.

Almost all of the flowers have curled up and turned brown, some of which are still kind of attached for the moment; does this seem like a good time to treat?

Removing the leaf galls by hand and disposing of them, as morz8 recommended, is all you need to do at this point in the year - it's too late and probably unnecessary for treatment with a fungicide. Lacebugs are another matter as they can produce several generations over the summer. Safest treatment option is insecticidle soap sprayed thoroughly. This is a contact insecticide, so follow the manufacturer's directions. There are also systemic insecticde sprays and drenches that need to be applied only once. The leaves showing lacebug damage will not recover, so next year treat before or just as new leaves are emerging.