Where do you live in GA? I'm in Atlanta and my Poms are in the ground and they are now fruiting. Plum trees are hardy, we grew them in Canada, many olives are hardy here. I do not grow many plants that are not hardy (no window space for winter).
If you choose to grow many tropical plants you'll have to water frequently (hard during water bans) bring anything in that is truly tropical (it will die in winter while not being hardy).
Make sure your pots are HUGE, big enough to old more soil than root ball so that your pot has space for soil to hold water and nutrients. Fertilize frequently (water leeches the fertilizers), every few years you either have to go up a pot size or two, or root trim your plants to keep anything large an appropriate size for the container. That is a lot of work for what might be little reward (small fruit plants).

Stone Mountain, GA ... I think my biggest fear is when frost in here .. just when I forget to bring the plant in, it come down badly .. (and I have lost plants in the past).

(Also I have tried some plants on my backyard, (Bougainvillea for example) died after a bad frost.

I've been thinking about planting (open bottom) pot in the planting whole which might protect it from root freezup (well, as much as possible)

Correction: Planting Hole (whole was a typo) ..

Also, I was curious how Zoo Atlanta have some tropical plants growing in Atlanta weather? I wonder what's their secrets?