The problem with reusing old soil is that it breaks down. The ideal potting mix should have good aeration created by solids in the soil(wood chips, perlite, ect), it will usually have peat for volume and water retention and then some composted nutrients(manure, ect). The organic soils I have seen at home depot have all been terrible at producing those ideal traits. They are usually very sandy and when wet become muddy. If you created your own then that's fine. The way that I reuse soil is to remove a few inches off the top(get rid of pests, pathogens) and remove the stuff at the bottom and sides where the roots have created a hard shell when tipping the pot over. The stuff in the middle can be reincorporated with new amendments or new potting soil, about half.

If it still looks and act like good potting soil as in friable you may reuse any of it as a growing medium. You are just looking to hold water and allow oxygen.
You just need to apply some means of fertility like compost or fertilizer.

I use pots the way you have. What I do in the spring, is remove the top layer of soil, and dump soil into my wheelbarrel. I then add fresh soil - organic Gardeners Gold something like that, - nothing like the junk at the big box stores, and I also mix in some cotton burr mulch. Toss it around and then replant the pots. I also wash the inside of the pot with a bleach solution to remove any crud.

I am growing potatoes in a trash can this year. When I talked about re-using the dirt next year I was told that was not a good idea. I also have garden kit raised beds where I re-use the dirt every year after adding some compost. What is the difference between using the container dirt and the dirt in the raised beds?