I don't know if there is actually a means of shutting off the oil completely, but you could try adjusting it all the way to the minimum setting (if it's even adjustable) and then see if it helps.
Alternatively you could empty the oil tank (back to the jug using a strainer funnel) and store the saw inside a heavy duty trash bag (to catch the dregs).
Another trick to try might be to position the saw so that the chain bar is on top, and parallel to the surface the saw is resting on. This way, the oiling orifice will be higher than the tank.
You must also consider that the oil tank or pump and plumbing channels might be leaking and if the saw is stored in a place subject to large temperature changes, the oil might very well be getting out due to thermal expansion (cold periods followed by warm periods).

Thank you for your ideas. I was thought there was something that I had missed. I have had the saw for about 15 years, the original box is saturated with oil. I was thinking about making a wooden box for it but wanted to solve the oil problem before I went to that work.

Oh Well the cardboard box still keeps it from gathering dust.