It could be the soil, lighting, and not having consistent fertilizer on a monthly basis. Citrus need food monthly because fertilizer drains out every time you water.

Most leaves look young and small. What happened to the older leaves?

The older mature leaves which must have fallen from your tree at sometime earlier would have been the fruiting flush for this season. The new leaves now on the tree are a foliage flush, not a fruiting flush. However, this foliage flush will become the tree's fruiting flush next season. Because your tree lost most all of its foliage, the tree used up all of its carbon reserves developing the current new leaves, and does not have enough energy reserves to now fruit. Your tree will do its part to fruit with the next fruiting flush *IF* you as the grower do your part.

I appreciate all the comments but still at a loss with some saying fertilize monthly and others saying too much nitrogen. And yes all the old leaves turned limp and fell off. The new small ones are yellow spotted so don't look great imo. I want to do my part, but still not sure what that is.

A close-up of the new leaves might help

check for spider mites or scale. "Too much nitrogen" means (for a Meyer) a fertilizer that is 3-1-2 NPK ratio; a higher N is too much.

Kevin, young citrus leaves always have a yellowish tint to them. As the leaves grow and develop into full sized leaves they will darken up on their own. Until a new citrus leaf is full grown it does not supply any nutrition to the tree. It is only after the leaf finally reaches its full size does it then begin producing food for the tree. Your tree seems to be coming along fine, don't worry your tree to death.