Our fireplace was brick and needed to be cleaned as well. My husband used a concrete cleaner he purchased from home depot and a big brush. It did work well.

When you say removing the panel and opening up the room are you taking a wall down? Replacing the paneling will make a huge difference!

This room actually is set up very similar to my family room. I love the floor you have in here!

Good luck to you!