Nicotinia and tuberoses come immediatly to mind as well as things like lemon sage and other herbs.
Linda C

thymes are all aromatic, as are lemon and clary sages.

if your soil is light enough, lupines have one of the most wonderful, spicy fragrances on the planet- I've gone to great lengths to get them to adapt to my garden (or adapt the garden to them ;)

a dwarf lilac is a possibility, though it might be a bit much for them in LA...

there are a whole bunch of twining jasmines, jessamines, and things CALLED Jasmine that would do well in your area...

and the tuberose that linda mentioned makes the gardenias seem like wimps, I love them!

Heliotrope (zones 8 - 11) has lovely vanilla-ish fragrance that seems to complement rather than overwhelm other aromas - it's a bit old-fashioned but a lovely plant (and none of my slugs will touch it). Freesia is good (you either love it or hate it), but you may have to raise in pots because I think the corms need a couple months of chilling before blooming. And, of course there are all the lilies; the madonnas and most of the trumpet/Aurelian types have good fragrance and are very care-free; buy by name for other varieties: some are fragrant and some are not. Try to get a few bulbs of a daffodil called "Geranium" - an excellent spring fragrance - in my area it blooms before the hyacinths - and don't forget to plant the hyacinths this fall, next year you can plant summer annuals on top of them. Nepeta and agastache each have a some nicely scented varieties. My nepeta starts to grow just as the hyacinths are finishing, by the time the H leaves are yellowing, the nepeta is big enough to hide them. And maybe you would like artemis, although I find you have to gently rub the leaves to release the aroma.

After exhausting research on all the lovely plants mentioned, I guess I'm leaning towards Gardenia, tuberose and licorice mint. I think it has a nice antique-y feel to it and all of those should be perfect for that particular area. The only ones I might have to worry about being "plucked" would be the tuberoses. I'll just have to plant enough I suppose.

The single tuberose are recommended for the coastal south over the more common double. They bloom in mid summer for me - So maybe no school children? Not to discourage you on the gardenias, I grow them also and they are high maintenance plants. Your soil might need to be amended if it is alkaline, depending on which part of the state you live in. In general, New Orleans and the west side of the river is alkaline. I live in Southeast Louisiana where the soil is alkaline and has a lot of clay. I have added alot of compost, pine needles, etc. I feed my gardenias with acid loving fertilizers and this year for the first time applied epson salt with very good results.