A good assortment of Aurelian lilies is always necessary. Lilac is a nice late spring thing. Maybe a Jasminum officinale if you're in z7. Wisteria is quite fragrant during May here. A wide variety of the native azaleas like rhododendron prinophyllum, atlanticum, alabambense and canescens (and there are others that all bloom at slightly different times ). Remember that nothing will be blooming for the whole season: you'll need a variety of different plants to scent your garden during different times of the growing season.

Something I greatly miss than I cannot grow in FL that always had an amazing spicy fragrance was Korean Spice Viburnum. Can't forget Peonies either.

I smelled Clethra alnifolia 'Hummingbird' at a local garden in Tulsa yesterday and really liked the scent and of course now I want at least three. :)

http://www.missouribotanicalgarden.org/PlantFinder/PlantFinderDetails.aspx?kempercode=a340