You should post in the NJ forum but here are a few possibilities.
Atlock Farm is in Somerset. 545 Weston Canal Rd, Somerset, 732-356-3373. It is a nursery with a nice display garden.

Frelinghuysen Arboretum is in Morristown.

http://www.arboretumfriends.org/

Then there is Willowwood which I haven't gotten to yet

http://www.morrisparks.net/parks/wwmain.htm

Well Sweep Herb Farm has a display garden

http://www.wellsweep.com/

There are botanical gardens within a reasonable distance in NY and PA.
NY Botanical Garden
http://www.nybg.org/
Wavehill
http://www.wavehill.org/home/
Longwood Gardens
http://www.longwoodgardens.org/

Here is a map at posted by Rutgers Cooperative Extention Center. Not all listed are a real gardener's destination - such as the Pequest Hatchery. But there are a few to look into. New Jersey Botanical Garden in Ringwood would be a drive from Warren but a good day trip too.

Here is a link that might be useful: Rutgers Garden Map

Thanks!

Dogridge,
Your post on the NJ forums say Warrenton, not Warren...I am not familiar with Warrenton. What part of NJ is that?
I didn't post there so I wouldn't scrowl it down.