Whoa; that's a tiny image!

Aphids would be on the leaves. And if you had root aphids, they're too sluggish to run around.

I suspect you have springtails or some other moist-soil-inhabiting critter. Rosemary drops leaves when roots are too wet.

Those things look like they have legs and maybe antennae. Springtails are just one of the possibilities.

Here is a link that might be useful: {{gwi:1193869}}

Perhaps this link wil be of some help in determining whether these are Aphids or not.

Here is a link that might be useful: About Aphids

The things that our original poster describes are in the soil and come to the surface when he/she waters. After scurrying around for a bit, they return to hide in the soil.

And what beetle are they the larvae of? Don't forget how tiny these little things are.

We've had other questions recently with images of very similar insects...they appear to be the larvae of springtails, as jean suggested long ago.