If you have tomato fruits already, I am envious. I'm only planting my seedlings today!

Did the unfruited part of the cluster have flowers? I would guess that if they flowered but did not fruit, it would be a pollination problem. Like possibly a temporary problem with heat or rain? I think if it was "energy", you would have all the flowers fruiting but they would all be a bit small.

The earliest fruits can be a bit iffy!

What have day time temps been? Top blooms in the cluster open first so perhaps cooler part of the early day. Then as the others further down bloom, higher air temps or hotter winds could prevent pollination. Keep in mind there is a somewhat narrow range of air temps for viable pollen.

What fertilizer are you using? Higher phos (middle number) is needed for blooms and fruits if you think they may be nutrient deprived but as yardenman said, I'd think you see it throughout the cluster.

Cherry plants on the other hand sometimes seem impervious to problems and power on when all others slow down so you can't really compare them. ;)

Dave