Did you put fertilizer in the dirt when you planted it? I've read that it's best not to fertilize new trees because most trees fail to over fertilizing when planting.

Only the compost and manure that was mixed the ground soil. We have a bit of clay in our area. Planted it high on a pedestal of dirt and mounded soil around it. No dirt was added above the pot soil. We did this so it should a good chance of not drowning.

The tree is trying to establish itself and develop a root system. It can't do that and support fruit too, so it has aborted the fruit. A small amount of dropped yellow leaves is a natural occurrence. Keep it well watered.

MrsG

Has thread going on leaf drop I'm thinking its sour sap causing leaf to turn yellow very mild case of sour sap your tree has falling fruit same as leaf they in coming days fall off, I think normal shock that on trees all most every year warm days in winter sap rise in trunk and sour what stop this rise sap trunk protection from sun paint are any cover from Sun. Link for more info on sour sap..

Here is a link that might be useful: Sour Sap.