no... I don't mean to be a downer, but if they can actually live in an east facing window, which you will be limited to low light herbs, those are WAY too small.
For example, mint, mint would start out that small, but would grow to fit a much bigger pot....as would other herbs, they grow.
you don't happen to have a doorway facing another direction where you could put a bigger pot? Most, not all, but most culinary herbs like a full day of California's sun.

those are unfortunately my only windows. I live in downtown San Francisco.
oh yeah and that picture is just an idea of what I'm looking for, there might be something similar that is specifically for plants that could hold bigger pots. I was just showing that pic just for the idea of a spiral shelf, not to show what size I want the pots.
Anyways, it was just an idea, I could always just get a normal planter box or something.
What if I use artificial light as well as the light I get from the window?

I dunno... San Francisco is hard place to grow plants that like full sun in the first place. I know... I lived there for 15 years. Orchids do great! :o)
Medicinal herbs might do well, due to a lot of them being forest and shade lovers. If you spent the $$ you could get the Metal halide lamps that might get the plants to grow.
Do you have one of those bay windows? That might make it possible... specially if you lived in like South of Market or other sunnier locations. SF is tough... I am sorry.

Thyme would probably work. Mint would work if you planted one sprig & kept re-planting it. One sprig of culantro/pot might work. I have all of these growing in low light areas  they get no direct sun. They might get leggy, but if that is your only growing option, why not try it?
You can also supplement your light, if needed.
If you do get a pot like above, you could also get a special bit to drill the present holders. A ceramic bit might work but you would need to check with a hardware store.

If you keep a fan on them indoors at night time they will not so leggy, but you also if wanted to keep it on in the day time it would keep it even more shorter.

Those drill bits are costly though,but Im thinking about the ones for glass, but still if ceramic ones are half as much I think their still expensive---- If you did do that practice first.

Chives would probably do fine in the situation you describe. 'Profusion' is a variety that does well potted. I keep mine in an east-facing window all winter long, and it grows like mad. Any of the broad-leaf thymes would probably do OK, too, since they are pretty tough plants. They would get leggy, but would then hang down and might add interest to the display. I also have Mexican culantro, Italian/oregano thyme, Rau Om and Vick's plant (in a hanging pot) in the same window and they are doing fine.

If you love the look of "the spiral" (couldn't resist), there are a lot of pieces made for multiple 4" and 6" pots. Search "plant stands" on any good engine. An added bonus to these is that they can be moved outside if you have even a teeny balcony.

Galium odoratum (sweet woodruff) would be another good idea. The flowers are used for May Wine and smell like heaven. One starter plant will fill out a pot in no time. Rosa chinensis (angel roses) might be dicey but worth a try. Ditto recommending Profusion chives. It's a gottahave.

Martie

Downtown? I don't suppose you could set up a small raised garden bed on the roof of your building? Four boards of lumber nailed together in a square and filled with a compost-manure-sand blend would be heaven for most herbs.

Failing that, I'd say grow lights, but YMMV on those, and from plant to plant (I can say from my experience that parsley, rosemary and ESPECIALLY marjoram will thrive under grow lights; lovage and basil, not so much).

You can buy incandescent grow bulbs ($6-$10) that will fit any lamp (a clamp on is cheap, too) or use fluorescent tubes ("daylight" type won't drive you crazy) in a hanging utility fixture ($10-$15 at a hardware store). Plug into an appliance timer ($6-$10). Set timer to come on 1-2 hrs after sun passes by and continue for about 10 hrs per day in winter and early spring. To offset the cost of the grow lights, start your plants from seed ($0 if given to you by a friend - $2) vs. plants at $4-$10 each.

I found a impatients grow kit at Menards, that I planted my herbs in instead of the impatients. It has a nice stand, that is space efficient,with 5-5in clay pots.I live in Indiana and have a grow light that I bought from www.seedlover.com. My herbs are doing great. If you haven't found anything yet hope this helps.