Thank you for the analysis!

Is there anything specific I should be doing now other than removing the fabric?

Should I fertilize or how do I know when to fertilize?

Rhododendrons and azaleas seldom require much in the way of fertilization. A light early spring application of a fertilizer formulated for acid lovers (like the Holly Tone) is often used just to assure sufficient nutrient availability in slightly to moderately acidic soils, which tend to hold back specific nutrients. But don't apply when the plant is stressed - like it is now.

Best thing you can do is make sure the azalea does not suffer from dryness over the summer, especially during droughty conditions. A slow trickle from a hose right at the plant base is the best way to water. Mulch will help in keeping the planting moist.. No fertilizer is probably needed, but a tablespoon or two of Holly Tone just as the plant is putting out new top growth will do no harm. Unless the landscape fabric is badly tangled around the roots, this is really not a case of what is usually meant by a stressed plant. It has suffered considerably from a difficult winter, but does not seem to be diseased or damaged by physical injury. Holly Tone is very slow acting and unless badly overused is unlikely to harm even a stressed plant.

I don't know why that wouldn't be considered stressed. Abiotic or environmental stressors - like winter cold - can have extremely deleterious effects on plants.

"Temperature stresses can also wreak havoc on a plant. As with any living
organism, a plant has an optimal temperature range at which is grows
and performs best. If the temperature is too cold for the plant, it can
lead to cold stress, also called chilling stress. Extreme forms of cold
stress can lead to freezing stress. Cold temperatures can affect the
amount and rate of uptake of water and nutrients, leading to cell
desiccation and starvation. Under extremely cold conditions, the cell
liquids can freeze outright, causing plant death."

Fertilizing a stressed plant is usually a wasted effort. The plant is simply not able to absorb and assimilate nutrients in the same efficient manner it can when not stressed, so most of the fertilizer goes to waste. And fertilizing at that time can also increase stress, complicating recovery.

gardengal: I suppose you could say anything that adversely effects the well being of a plant is some form of stress. Extreme cold certainly falls within that category, but once temperatures moderate the stress is gone unless there has been permanent damage to roots or some other plant part. I have no idea what the OP's geographical location is, but the photo is pretty much what unprotected evergreen azaleas look like every spring in large parts of Z6 and Z7. This past winter has been even harsher than usual in the East and Northeast. If we took this kind of winter damage as a signal not to fertilize, we wouldn't ever use fertilizer at all. Two tablespoons of Holly Tone are not going to harm this azalea and may actually help. It's a whole different gardening environment in parts of the country routinely subjected to near 0 and below 0 temperatures, often accompanied by high winds. Very different from the benign climate of the PNW.

It sounds like my azalea will have winter damage every year if its not protected as mentioned with some sort of a cover.
I'll try adding a small amount of holly to it, but is it the right time to apply or should I be waiting till late spring?
Do you think Ill be looking at a green bush by this summer or will it take until next year for this bush to fully recover?
Also, for further care, should I prune off any branches that have the brown leaves?

Be patient on the pruning. Branches may well recover and put out new leaves even though they are looking pretty woeful now. The dead brown leaves will drop off on their own. Add the fertilizer if you want to use a little bit when the new growth is apparent. Cut back any obviously dead branches in late spring. Cut back to a point where you can see green cambium under the bark. Some winters are mild enough so that damage does not occur, but for evergreen azaleas exposed to a lot of winter sun and wind, protection is a good idea. It does not have to be elaborate. The remnants of the Christmas tree can do it. Light weight fabric intended for frost protection can be draped over it. Burlap stapled to some stakes works. The whole idea is to provide shade and wind protection. Nothing can be done about the temperature, but it's really the combination of cold, sun and wind that does the damage. Chances are very good that it will recover this summer.

Great, thanks for all the great advice!

I am in the SAME boat. Brown bloom-a-thons. This 2015 boston winter really browned my little bloom-a-thons. I covered them completely with a styrofoam cover in the fall. I also wrapped my $100 6' pencil plants ipex with burlap. All dried up and leaves are just falling off. What more could I have done???? I scratched all of them with my fingernail and they are still green, but naked! Should I wait it out. Last year it was flower less hydrangeas and this year it is dead evergreens... Many plants split in half.. It is just a sad Spring.

WHY ARE BIRDS EATING ALL! THE LEAVES BROWN RED COLOR NEVER FLOWERS AZALEA BUSH?????????????..?..3 YR PLANT

PLANT HAS NEVER FLOWERED 😟😟😟😟😟😟😟😟😟

BIRDS ACTUALLY DEVOUR ALL THE LEAVES😟😟😟😟😟😟

Ruffed grouses looove to eat the flower buds and so do wild turkeys, deer, squirrels and bunnies. You can try bird netting against the birds and Plantskyd against deer.