Sounds like the pupae of one of the hornworm caterpillars. Check out the attached image and see if it matches up with what you've observed.

Here is a link that might be useful:

Yes, that is it. Would this be harmful to a garden? Maybe eat the roots?

The pupae do not eat roots (or anything else, for that matter). Once the soil warms up, an adult hornworm moth will emerge from the pupal case and crawl out from the soil to fly away.

Was this location the site of a vegetable garden in the past?

The hornworm moths tend to be fairly large and many of them quite beautiful. Certainly interesting. The caterpillars of those moths feast on a very wide variety of plants, depending upon the species.

All caterpillars are plant eaters, but few of the hornworms present a real problem for gardeners.

I have a few varieties of hornworms. The tomato hornworm which eats members of the nightshade family and Catalpa worms which only eat my Catalpa tree. They can completely defoliate a plant or tree but the Sphinx moths that they turn into are so cool. There's nothing I can do about my Catalpa tree but I remove the Tomato hornworms from my brugs and daturas and have a little tomato patch just for them to feed on. The caterpillars are usually fairly large and come in different color patterns but they all have a little horn on their butt.
Karyn