A sharp spray of water is all that is necessary to wipe out aphids, although that may need to be repeated evey 5 days or so to clean off newer generations of aphids. The eggs that became the aphids were probably in the starting soil you bought.

Repetition and persistance. You CAN spray with an insecticidal soap, but it's not needed. A spray bottle, set for "stream" might do the trick, and be easier then moving them to the sinkk and back. I have heard of some people squishing them, and someone else told me she wraps packing tape, sticky-side out, around her hand and swipes at them with that....

Aphid eggs won't be found in the soil. They came in the windows or doors. ;-)

You can safely use Neem oil indoors, as another pro-active tool other than water. It works great for aphids and is a non-toxic indoor plant spray. Some people complain about the odor (sort of onion-y) but its better than a host of aphids!