Anyone have thoughts about this? The hole is now 4' diameter x 2' deep, starting to get to some soil in the deepest part, but the sides are still mostly rubble/stone.

I will go out on a limb here, pun intended! I think that size hole will be adequate. Just make sure the hole doesnt become a catch basin, it needs to have very good drainage.

Well, you can grow trees in pots which are smaller than that so it will grow in the ground. It will be limited because of the lack of soil but since you bought a dwarf that sounds like what you are going for...

Apricots are supposed to grow fine in rocky ground, so that's good! I don't know if that applies to to dwarfing rootstock, though. If you've hit soil in the bottom of the hole, I think you'll be fine either way, as long as the drainage is good. I don't think the rock 'walls' will hurt a thing, except maybe needing watering more often than if all that were also soil.