What is your question, how far to put it away from the fence (is it a wood fence and you're concerned about rot?), or whether brick is the right material to use, or something else?

Considering you're in Texas, is the area in the shade or sun? If it is in the sun keep in mind that the brick will heat up and could create a micro climate that is hotter than the surrounding area. In the right conditions it could cook your plants.

Yes, it is a wood fence. I was thinking if I put the dirt against the fence it will rot.

Good point about the brick. I am going to use some kind of border, a stone of some type. The bed will be in full sun.

Brick or stone would both work. You are wise to keep it away from the fence a few inches. Plant your heat tolerant plants near the brick, actually you'll need all heat tolerant plants in Texas, no? Good luck!
Tina