Maybe the tree is not happy. Tell us more about the tree and where you are located. any pics?

Also add feeding regiment and type of food. How often. What is the dirt its planted in like. Is it in a pot or in ground, How much sun does it get.

I think the tree is a ruby star grapefruit. I feed it all kinds of stuff like foliage pro, citrus tone, fish emulsion, Azomite, blood meal and bone meal not all at the same time of course but probably every 2 weeks. It is on my back deck getting full sun and it is in a pot. The tree is four years old. In the winter it is in a heated garage with a grow light and a humidifer. All my citrus trees have lost a lot of leaves and I thought sure they would grow more after being outside but summer is almost over and they didn't grow any. It was in potting soil and did nothing, transplanted to Al's gritty mix and they didn't look so good so I mixed it half and half of potting soil and Al's gritty mix which they are currently in. The leaves look droopy. I don't want to lose them but I don't know what to do. Sometimes I give them potash and Epsom salts. I will try to upload some pics.

P.S. I'm in Alpena, Michigan, zone 5

Thank you Tom, I will go out in the morning and snip the fruit off and try to find them some partial shade

I don't think those trees are well established enough or have enough foliage to sustain many fruit at all. They need to grow another year and put on some size. I think the fruit crop is keeping it from growing foliage and the lack of foliage is keeping the fruit small. I'd probably eliminate most of the fruit.

Thank you for your help. I just went out not ten minutes ago and snipped off all of the fruit. If they have been on the tree over 6 months and are no bigger than a pea then they wouldn't be any good anyway. I certainly appreciate everyone's help. Thank you, Cathy