Azaleas can get spider mites (you should see some fine webbing on leaf undersides when closely inspecting) -

but I would suspect azalea lace bug from your description of black specks

Here is a link that might be useful: Lace bug

I agree.

Azalea Lace Bugs, Stephanitis pyrioides: Adults are about 1/8-inch long. The body is pale yellow. The lacy wings (very distinctive) are held flat over the back and are transparent with two dark spots present. The nymphs are black, spiny and smaller than the adults. The eggs over-winter partially embedded in leaf tissue. The eggs hatch in May. The nymphs mature into adults in June and lay eggs during late June and July. The second generation of nymphs appears in August. The over-wintering eggs will be laid when these nymphs become adults. Adults and nymphs feed on the undersides of leaves by piercing the leaves with their mouthparts and sucking the plant juices. This causes a mottled, silvery or white discoloration, known as stippling, on top of the leaf where the chlorophyll has been removed. The undersides of leaves are covered with dark brown to black, sticky spots of excrement. Plant rhododendrons in shade to maximize the activities of beneficial insects.

This insect hatches early in spring as the new foliage begins to mature and its numbers may build to damaging levels with successive generations. Lace bugs reach their peak in late summer and do their worst in sunny, exposed sites. Spiders are important predators of lace bugs and since they shy away from sunny, hot places, plant your azaleas where there is some shade. Insecticidal soap, horticultural oil, or a systemic insecticide may spare your azaleas the damage if applied in spring when the first hatchlings are noticed. Care must be taken to spray the lower surfaces of the leaves where the lace bugs live. Moving a plant to an area with more shade may alleviate the problem. Lace bugs are more prevalent on certain varieties. The following azalea cultivars have resistance to azalea lace bug: ´Dawn,´ ´Pink Star,´ ´Ereka,´ ´Cavalier,´ ´Pink Fancy,´ ´Dram,´ ´Seigei,´ ´Macrantha,´ ´Salmon Pink,´ ´Elsie Lee,´ ´Red Wing,´ Sunglow´ and ´Marilee.´

Similar damage with black specks on the leaf underside is due to thrips.

Thrips are found in dry climates like on the West Coast. They are seldom found in warm moist climates like the East Coast.

Thrips, Thrips imaginis, are characterized by a silvery white discoloration on the leaf's upper surface and silvery on the bottom with small black frass specks. The damage is similar to that from lace bugs. Thrips are a problem in warm and dry climates like California and New Zealand. Most of the standard insecticides and insecticidal soaps can be used. Control for thrips whether an insecticide or insecticidal soap is a contact control and must be applied on the adults. This is most likely done in May in California when they are visible. Locally, biological controls may also be available. It is best to remove infected flowers.

This insect is a seasonal pest, causing most damage in mid to late summer. Thrips are preyed upon by ladybirds and lacewings but these predators cannot control thrips in plague situations.

Here is a link that might be useful: How to grow azaleas and their problems.

Thanks to all so much who responded!!! - OK, I think from the hard spots and the number of them, this IS the lace bug, and what I am seeing is generation after generation of these things. The artilce link was really helpful. I am not a big pesticide user, but will pull out the heavy nasty stuff if I have to, so these bushes are going to get nuked with some Orthene ASAP. Thank you all again I want to save my bushes, these are some nasty pests.

This will be a continuing problem unless you move the plants to more shade. There are other varieties that can take more sun and not have lace bug.