Find a small branch and cut it. If it is brown throughout the center of the branch, cut another. If that one is brown inside, as well, your Osmanthus is probably dead. If it is still alive, please remove it from the compacted soil it is in now, and make a gritty mix for it, if you are keeping it containerized.
I grow my Osmanthus outside, in the ground, so I cannot recommend the proper potting mix for it. I do know that they seem to grow best containerized in a very well-draining medium.

The number one killer of containerized plants is over watering.
The second is a poor draining mix.
The third is a tie between too little water and over-fertilization.

With Osmanthus, many folks also want to park their containerized plant far away from light sources (be it grow lights or a good west/south facing window. Osmanthus LOVE light and do best with half a day to a full day of direct sun.

Hello! I'm so sorry you're having issues with your Osmanthus fragrans (Sweet Olive).
I am not an expert when it comes to this plant since mine always look terrible and I think it's going to die but it holds on and then produces the most gloriously scented blooms so I keep trying to make it happy.
Anyway I would repot it into a faster draining, lighter potting mix like the 5:1:1 mix.
I did a search and people who know way more than me when it comes to keeping sweet olives happy say osmanthus prefers the 5:1:1 mix over the gritty mix.

-Robert

Here is a link that might be useful: GW tips for sweet olives

Yesterday after reading the first reply, I took the plant outside and just a little wind blow another 10-15 leaves off the plant, I moved the plant out of pot and saw a layer of white roots surrounding the root ball, felt a little better, it may be able to make it. I was really afraid of bothering the root ball, so I just put it in a clay pot (might be half inch bigger than the original plastic pot, and filled the gap with perlite, hope it can survive, so far it may have about 15-20 leaves left.
Robert (sorry got a typo o in the first post, as I was so upset), your plant gave me a little hope, wish mine good luck as yours. you are like a celebrity here in my mind.... I looked at some 5-1-1 mix, based on your experience, what is the bark - pine bark fine or hardwood bark? please give me any advice you have, I feel myself is so vulnerable for the upset plants, as I just lost my 10yr old cat four month ago, and I am still in grief process, and really wish all my plants can grow well to cheer me up..... thanks for AuroraWA as well.

I'm so sorry about your cat. I do hope your plants can cheer you up a bit.

And you flatter me. Trust me I've killed lots of plants and haven't been successful with lots of plants as well. ;)

I use pine bark. I screen Hapi-Gro Landscapers Mix that I buy at Lowe's.

If you can't find that locally you might try posting in the container gardening forum.

Good luck!

-Roberto :)

Here is a link that might be useful: 5:1:1 tips on the container gardening forum