I've had a hard time overwintering rosemary indoors. The others are winter hardy in my zone so they stay outdoors. Lack of light (and I have an old house with lots of windows) and low humidity indoors being the major winter problems for me.

As far as soil for the first three, I would try a bonsai mix. It drains well and is on the lean side. These plants will resent potting soil, too rich and not enough drainage.

FataMorgana

You might try the rosemary in a separate container. The container growing forum has some excellent advise for making your own well-drained long lasting media for container growing. It can be a challenge to obtain all the materials but is well worth the effort in the long run and you will never go back to "regular potting mixes" again. Great for all but the wettest loving herbs or plants you might container grow. Lots of sun from a window and well-drained and not to rich on your mix (forget the compost IMO) and you may well be good to go.
Good Luck and Happy Growing David

Sand doesn't behave very well in containers, either. Like compost, it is best left to the gardens. You can create your own terrific mix by adding a hefty portion of pine bark FINES, Turface, and perlite to a bit of your own potting mix.

I use such a mixture (more or less) for all of my container plants, from bonsai to annuals, with great success.