I wonder if that's damage from citrus rust mites. Maybe you can spray the foliage thoroughly with an organic spray that contains spinosad. I use Captain Jack's Deadbug Brew, and I mix in some neem oil and a bit of natural dish soap.

Could be that your tree is in marginal soil (the heavy clay) and was getting along ok until you had some weather conditions that changed the soil conditions for the worst. I have some lemons in a very wet corner of my yard that have done surprisingly well but now I'm seeing some signs of stress. This past winter was REALLY wet here in SE Texas and now those particular lemon trees have not put on near the blooms that a tree in a higher part of my yard have put on.