Aphids could be blasted off with a hose - but could be difficult to do indoors :(

Soap may dissolve protective wax on leaves - that's what it is made for. If you want to use soap do as Dave suggested.

Spraying with just water will work well, but they may keep coming back regardless what you use. Will you put them outdoors once it's warm enough? That helps...

If spraying with anything but water, don't do it in full sun (after spray dries off, it is OK again for plant to be in full sun).

or you can order some ladybugs … and witness the carnage ...

Dave - soap solution will kill soft bodied pests like aphids. It is a contact killer but it would definitely will be better to use something castille soap or insecticidal soap made specifically for plants. The safe ones will say potassium salts of fatty acids. Some plants may still be sensitive to it and should be used with some caution.

Alcohol solution is quite safe. It stinks up the place for a short time. Inside the house it will not leave unsightly residue on surrounding stuff - which is a big plus.

Moinrules: If you have not seen any damage so far then you are OK. Usually the concentration used is 1 tbsp of soap per gallon of water. I usually use 2tsps per gallon and is quite effective. It has to be a real soap and not the usual dish detergent. Usually organic ones are made of real soap - sodium or potassium salts of fatty acids. Try to go for the one that says (if it does say at all) potassium salts. Castille is sodium based, BTW.

Thank you everyone for the suggestions.

Dave: if I understood correctly, you proposed two solutions to the problem:

1. Castile soap mixed with water application, followed by neem oil application.

or

2. 50/50 Alcohol/Water solution application as a cheaper alternative.

For 1, would I be putting neem oil directly on the aphid infestation, or making some spray out of the neem oil and applying it?

I see you changed your id here. Anyway I will attempt to give a partial answer.

Concentrated Neem oil is used as a spray mixed with water. Or you can get a Ready to Use type from the store. Homedepot/Lowes usually have them. This tends to be very expensive if you have a lot of plants.

Or you can get concentrated version of neem oil and dilute it with water yourself. In this case the package may state that you also use a few teaspoons of liquid soap to help the oil disperse well.

I say you go the alcohol route as it is the cleanest and very effective.

Sounds good, will try that next; I still have some aphids left from my previous novice experiment.

You want to use a soap with long chain fatty acids. Soap is a pretty broad term... Insecticidal soap is just a specific one chosen to minimize plant tissue damage with its longer chains.

Virtually all soaps are phytotoxic. Care needs to be taken and each subsequent application typically results in increasing amounts of damage.

Quick update: I used the alcohol-water solution a couple of times over a period of one month. It sure helped to control the aphids population, but they kinda kept coming back. I then removed an overgrowth of violas from one of my layered containers, since I noticed very concentrated aphid infestation on those leaves. After that, the aphid population has drastically reduced. I hope they'll all eventually die off.

I don't know where you live - could you put plants outdoors? Aphids will probably keep coming back indoors. There are at least predators outside, and you can blast them off with a hose when watering.

I live in Northern California (Zone 10a). My plants are in my balcony outside. Like I mentioned, the situation is much better now; will update if things become bad again. :-)