Some other directions/ideas :) --

Ed, do you find a fireplace in front of a window a bit odd???....Bobbi

bobbi, I have seen gas fireplaces installed and surrounded by windows. It is a great look, but, as you can see, the venting and firebox are exposed in the yard. Thus the picture on the mantel.

Hard to imagine what tearing out the wall will look like without pics of the other side. The room does look narrow so opening that would probably make a huge impact. Lots of structural implications though.
I like olldbobbi's fireplace suggestion. Painting one of the walls an accent color that compliments the furniture would be fun - maybe a soft blue. You could also add a floor rug and change the ceiling light fixture to dress it up. Maybe something like this.

Great ideas. On the other side of the wall is a kitchen-- see pics. Thinking of fireplace surrounded by built in shelves. Love the light fixture! Ok-- can't attach pic right now, will have to attach later.

So you're wanting to open the room up like this? I think it's a great idea!
Depending on the layout of the kitchen, think about a double sided fireplace.

Love the light fixture - where to purchase?

An open kitchen that opens to a family room and vice versa is really nice. Since you're not using this as a TV room it could be perfect. I love the double sided fireplace idea too if you can make that work.
Here's the link to the light fixture.
http://www.lampsplus.com/products/double-drum-16-inch-wide-bronze-organza-ceiling-light__x4274.html

The brick surround is too big for this room, so I would build a drywall box around it and install a traditional stone and wood fireplace surround.