Sounds like aborted fruit from a flower that was not pollinated?

In this case I'm pretty sure it was pollinated since I've seen what happens when they aren't (from other flowers on the tree)

I haven't ever fruited a cherimoya, but I have seen this happen in Sugar apple, atemoya and most recently, rollinia. There is a definite difference in a flower that has no pollination and one that at least has some level of pollination. As far as I observed, an unpollinated flower goes into male phase, sheds pollen and then drops with absolutely no fruitlet appearance. When pollinated, the flower doesn't go into male phase. Instead, a small fruitlet forms and then, eventually starts gaining size until it is mature. There seems to be a fair amount of fruitlets that do not fully develop. They get to a small size and then turn black and drop. I recently experienced this with 100% frutilet drop on my rollinia that was blooming for the first time. Intially, I had about 10 fruitlets that formed initially. The loss of all ten was somewhat of a mystery to me. We did have a severe drought at the time but I provided some additional hydration. In talking with friends that have fruited rollinias previously, they had a similar experience with the early blooming. The concensus seems to be that the immaturity of the tree is the main factor. It was suggested that the tree must reach a "critical mass" of blooms that allows for the dropping of a number of these frutilets that, for whatever reason, are doomed to turn black and drop. So, I am not sure that this is the full answer to the problem. Nutrients, heat, moisture may all have some responsibility in this problem. Observations continue.

Harry

That might be it, I've heard that it needs to be at least 8 feet tall for it to be able to support fruit, so I might just be under the min threshold.