Grab a pick axe and start digging. You'll find more than enough rocks to make a border. Cultivate your flower beds to the depth of a few feet and you should be able to build a fortress with what you've pulled out. There is no soil in this part of the country, only rocks and stuff in between.

If you're not feeling up to the labor, I'd recommend calling your local highway department and asking them about telephone poles they may have replaced over the past year. The poles are treated with creosote so they'll last a good while.

Thank you Shakazulu.
I've heard of using old railroad ties but never telephone poles,that's a cool idea.

Beware of creosote and other preservatives in the garden. They leach cancerous poisons into your water system and food, and they make the garden smell bad.

I have to laugh at the places that have a sign that says "Landscape Materials" in Ulster County. All they are are piles of bluestone and other rock rubble you can pick up on the side of ANY road in the area. Drive through the country and you can fill your trunk. We are very special rock gardeners here!

Thought of another idea for a border, though this won't do much if you are looking to create a raised bed. My sister does a lot of work at a horse farm and gave me a box of old horseshoes. I stuck the prongs in the ground and overlapped the shoes to form a border around my garlic bed. I think they look rather nice like that.