We have had several threads on the Garden Restoration forum re both lesser known and world-class historical gardens worth visiting. Do a search of the GR forum and/or ask your question there as well.

I have enjoyed the plantings at Shelburne Museum in Burlington, VT.
The garden at Shelburne Farms was ok, but seemed like mostly annuals. Of course in the big picture, it is an Olmstead designed landscape. I did come away with a good idea which is to pay attention to the angle of the sun and how plant colors will catch it. I enjoyed Naumkeag because even though the plantings are small, they photograph big. I likedthe Mount too because they were restoring it so I got to see the "bones". And believe it or not, the Bristol Clock Museum because the plants all had a story and it's a do-able project.Hill-Stead is a nice example of Beatrix Farrand's work. I liked Berkshire Botanical GardensA LOT. Hancock Shaker Village has gardens worth looking at. And in Maine, everyone loves the McLaughlin Garden.