It probably depends upon where you live and what plants. I can control mint by simply not watering. :D But that's not an option for most people. Do not ever sink mint. You will be sorry. I'd put it on cement if I were you.
Yarrow is pretty invasive to me, fairly, mostly by seedlings, but it also just gets wider. :o) Hyssop isn't as invasive for me as it probably is for you, it's hard for me to tell as I live where it's really dry during the summer.
I try to plan for some invasiveness in everything I plant, just to be on the safe side.

I've grown Catnip in zone7 and here in WI and I've found that it usually stays in a clump...but it will reseed everywhere. You can always remove the flowers before they go to seed. The seedlings are easy to recognize and pull up.

In my garden, catnip stays pretty contained to one area (it doesn't crawl like other mints). I harvest while in flower, so I don't have issues of reseeding. My hyssop (hyssopus officinalis) is very tame and stays put. My giant hyssop (Agastache foeniculum) is another story though. It reseeds everywhere!

Yarrow creeps, but cultivated varieties seem to be more tame in my experience.

From experience, hyssop no, yarrow yes.

Aside from mint, one of the most invasive plants you can grow is evening primrose. It spreads even faster than mint in my part of the world (subtropics). Yarrow can become a thug, too. Speedwell can spread well with speed; St. John's Wort is such a baddie that it has become a declared noxious weed in some areas; nasturtium in my part of the world is fast becoming an environmental disaster; amaranth and perilla have absolutely no manners at all; and in my garden, even parsley can spread far and wide (it depends on where it's growing whether I treat it as a weed or a valued plant!).

Many people on this forum say that chives can be a bothersome plant, but I've never found that. At the moment I'm watching my soapwort carefully, because that can become a weed in some areas. Kotu kola and Brahmi are 2 plants that I would never dream of putting in the ground.

Borage can self-seed anywhere and become a weed - not a particularly attractive one either, although the flowers are gorgeous. The suckers of my elder tree are becoming bothersome to my neighbour over the back. Comfrey can become a weed if the roots are disturbed.

Many herbs start their lives as weeds - think of dandelion, chickweed, plantain, couch grass, cleavers, stinging nettles and umpteen more. In my garden, arrowroot is becoming a weed and I have to be careful to remove seed-pods as they appear. I grow it in the back garden, but there are arrowroot plants coming up in the front of the house, so it's one that needs watching.

I think the list of potentially troublesome herbs is much longer than most of us realise!

oh that evening primrose from HELL! any evening primrose, Mexican too.... it's a noxious weed here too. :(

Many of my herbs seem to be mostly well behaved. I think my heavy clay soil and the western NY state winters slow their spread whereas people in no frost zones probably have much different experiences. I find the very effective self-seeders to spread far more in my gardens than the plants that spread under the soil, like mint.

I worked with a person from Brazil who said tomatoes were terrible weeds that sprout up everywhere - you gotta remember tomato seeds do survive the human gut. And the folks here plant seeds indoors weeks a head of time, baby them along, plant them in their gardens, get some tomatoes, and the first frost kills them all. So what's a noxious plant for one is not for another. I think the invasiveness of any plant varies with your growing and garden conditions. Speak to local gardeners about their experiences with any given plant.

FataMorgana

I would like to mention that this is a great thread; if I had read something like this when I began planting herbs it would have saved me a lot of trouble! I am in Pennsylvania, USA, zone 6, with rich clay soil and good rainfall. Our summers can be quite humid, and my property has several mature trees, so never gets truly full sun.

When I started a little herb garden years ago I had no idea of the growth habits of any of the herbs, as I had never seen any of them growing in a garden (or anywhere else, for that matter). The references I was able to access at the time were little help. I returned to work shortly after that, and the next time I was able to give attention to my herb garden, the lemon balm/mint had completely overgrown all of the other herbs! It eventually seeded itself in little clumps all over my yard. I sometimes wish I hadn't pulled quite so much of it up, as it eventually subsided and practically disappeared.

I found myself planting more of it last fall, as I am drinking more herbal teas these days, and also to compete with Yellow Archangel (an even bigger mistake I made more recently).

I had the same experience with the purple-flowered agastache--"Korean Hysop," it easily gets two feet tall in medium shade and spreads by seeding itself all over the place--but it is easily pulled out and makes a good addition to herbal teas as well.

Bee balm, on the other hand, will grow for me, but has never been particularly invasive; on the contrary, it seems to die out after a few years.

Since I really don't get full sun, which catnip prefers, it has never done well enough for me for it to become a problem.

Thyme and sage will overwinter for me, but only in pots. Thyme is so tiny it is easily overwhelmed by almost anything else, and I never plant it directly in the ground anymore, as I always lose it. I have found it will fill a gallon pot or equivalent container if kept to itself, however.

Oregano will overwinter for me, but it has never become invasive; it is so low compared to many other herbs that it needs to be kept next to the edge of a bed, or in a pot.

Coltsfoot will really take off and quickly cover a large, cultivated area--it is viny--but it died out after a few years, and I really don't consider it a problem under my conditions because of that (except in a small bed with other things that you want to survive, of course.)

Lovage can get to four or five feet tall, with numerous leafy stalks. Even it can be shaded out by Angelica Archangelica, which develops wider leaves and can cast more shade. The Angelica A. I have in my garden right now was started last summer, and is about 3 1/2 feet tall, with leaves that extend out like wings for about 18 inches each. The central stem is about 2 inches thick.

Comfrey will shade out anything in a four to six foot radius. Its leaves and stalks will initially grow straight up, but then collapse into a circle around the plant, smothering anything underneath.

My most recent experience with invasive herbs was with Motherwort. I bought one small plant at Sturbridge Village a few years ago, and it spread through the back fourth of my yard by seed, even in lawn! It makes copious seed, as it gets about two feet tall, has numerous tiny flowers at every leaf node along the stem, and is a sturdy plant that gets spiny as it matures. It is not that difficult to pull out with gloves, though, and the young plantlets appear to be controllable by mowing.

Ref. the comment above about tomatoes--even in my climate zone, a popular little yellow salad tomato has occasionally come back the next year in my husband's vegetable garden.

Any mint here is very invasive--we have to pull it up by the handfuls, and even sinking it in the ground in a bottomless pot doesn't keep it contained.
Yarrow, especially the white, is invasive, but again, a few good pulls and the problem is solved.
Our Greek oregano will spread and spread, but that really isn't a problem because we use so much of it!
Marjoram spreads, but not so much as to call it invasive.
Rosemary gets HUGE, and of course overwinters in the ground. We just have to put it where it can spread out and enjoy lots of space, so it isn't for a little herb garden outside the kitchen door.
Garlic chives get bigger and bigger, and then the clump gets dug up, divided, and shared with anyone who will take some! Onion chives don't do too well for us--not enough drainage, I fear, so I don't know if they invade.
Nepetia, catmint, doesn't invade here, and sure is a pretty plant.
I'd rather have an invasive herb that we enjoy and spend a little time keeping it in reasonable bounds, than to forego the pleasure of tasting, smelling and "messing about" in the herb garden!

Sweet Woodruff spreads madly but it is easy to pull up and so attractive that I use it as a ground cover and underplanting.

I cant believe nobody has mentioned common tansy as being invasive. I planted Tansy 3 years ago and that single plant has reseeded everywhere. That single plant is a monster at 5 feet tall by 4 feet wide. I have many offspring from it, and those are starting to get to large and spred. They will sprout babies everywhere, Even in gravel. You have to pull it out the minute you see the babies in spring otherwise left to its own devices it becomes a huge monster buy summers end and the roots are harder to dig out.I have babies everywhere this spring and I went out early and pulled them all up. We moved the larger Tansy Plants in front of the Gas tank to help hide it. That was hubbys idea. We just cant let it go to seed anymore regardless of how beautiful the dried flower heads look in the winter. They are a butterfly and bird magnet to.

I had the same experience with the first Tansy I planted here in zone 6, too--it got enormous! For some reason (possibly the partial shade) it stayed in the middle of the bed I planted it in, making a nice fern-like backdrop for roses and other small shrubs. Maybe the shade from the shrubs contained it.

I planted some more of it in a wilder part of my garden last year--so far it is only about a foot tall.

I also planted some of the curly Tansy under a tree in my front garden. That is struggling alone in a fairly shady spot with sharp drainage and little soil, so even though it has been there several years, it is spreading very slowly and seems to stay at about eight inches.

(One of my favorite tricks is to plant a thug in a challenging position and see how it does).

This has been such a helpful thread! Thank you all for your input. I had forgotten all about asking about lemon balm, of which I have a few seedlings, so thanks for reminding me that it can be invasive. (I had read so, but I have trouble keeping track!) I'm also glad I decided to forego the tansy. It's recommended as a good bug magnet for companion gardening, but I read it was highly invasive and I just don't wish to keep up with a monster plant that spreads everywhere - well, not if it doesn't taste good or offer some useful medicinal benefit anyway. :)