We brought ours up from New Orleans where it had to be brought inside for coldest weather. Here we're 18 miles ne of Knoxville.
Big heavy pot held about 15 gallons of potting soil and supported a four foot tall, shrubby sweet olive.
I kept it alive for five years up here by overwintering it on our passive solar porch (taking up lots of water some weeks in winterf; a lot less in cloudy, dreary weather.)
It can be done, but in winter takes as much care as a pet to keep the water right.

ooh that's where we were living--in New Orleans--when Mom discovered it. That was in 1959. We lived on our sailboat in the West End Canal by the Coast Guard station on Lake Ponchatrain--ah fond memories. I think I'll just abandon the idea of getting one. My dwarf gardenia will have to suffice.

Hi, I have grown a sweet olive tree for about 2 years now! It is very slow growing, but so far has done well.. I have it in a border along my fence and so far so good! I am in Memphis.. so maybe try it? If you mulch heavily and put it with a little protection? Just a thought! jenn

I live in Ringgold, Georgia just south of Chattanooga and have been growing a sweet olive on my deck for 5 or 6 years. It stays outside year round. It has been very hardy for me. It is about 6 or 7 feet tall. It has gotten down to 7 or 8 degrees without damage. It is in a huge pot.

I am in zone 7a. It blooms mainly in the fall but also some in the spring.

Thanks everyone for all the comments. I was just wondering if anyone had ever seen a sweet olive at Lowe's or Home Depot? Where did ya'll get yours?

I got mine at the home depot a few years ago, but I see them every year, I got mine in the summer time! HTH! Good luck! Jenn

Ecolage has them. See link below. They specialize in Southern plants.
I ordered a night-blooming jasmine from them last year and was extremely pleased with it. Arrived as a huge plant (about 2 feet tall) with several branches and covered in buds which bloomed a week later.

Here is a link that might be useful: Sweet Olive at Ecolage