I'd tear it off . What stone veneer? What flooring? What is the feel you want in the space? I'd be rid of the raised hearth before flooring

If not removing the wood panels, I would paint the wood walls and the crown Moulding the same color as the walls. Not sure if there's a plan to paint door/baseboard Mouldings white or leave as is?

I see avery traditional dining table and why would you have a nice carpet on the floor in the midst of a huge reno? The wood walls are screaming dated so remove them they will still be dated painted . I have no idea what your style is or the stone or the floors you have planned . IMO you are going too fast without an actual plan for the whole space . Is that FP up to code ? Do you have to have a hearth ?

I hate the term ‘dated’ but diagonal wood paneling is super dated and doesn’t vibe with the rest of the room (works fine in some 80’s shed homes for instance). Get rid of it. Probably rethink the stone, most looks fake and is this decades diagonal paneling…

I wanted something neutral for the living room. A lighter stone veneer for the fireplace. I was planning on replacing the door. Also please note the tv, table, carpet clock was all left there we didn’t place any of those items there. We don’t need a hearth the fireplace is up to code.

Would you recommend leaving the brick instead of placing stone?

for me all different materials, plus the dark trim, white baseboard heaters and wall angles make the room feel chaotic - I would want to tone it all down and get everything painted the same color (ceiling included) once new flooring is down (with raised hearth removed) and everything is painted out - then I’d address the fireplace. Agree with @HALLETT & Co. on the stone veneer: it is everywhere these days and often in places it shouldn’t be

Keep the brick, remove the diagonal wood panels. Prime the walls with two coats of best white primer, then decide what paint color you want.

Can you box in under the hearth with brick or wood trim?

Otherwise paint the brick a very dark charcoal gray to downplay the brick buttressing at the floor level.

I wouldn’t do the brick just yet. There are so many options for ways to redo it.

I would start was removing the wood paneling and concentrate on painting the home first.

I would then furnish the room and this can help you figure out the fireplace style.

"I wanted something neutral for the living room. A lighter stone veneer for the fireplace. I was planning on replacing the door."

Neutral is a color....many colors. Depending how you use a color, almost ANY color can become a neutral. It needn't be oatmeal, or putty, or gray, or beige.

Style, conversely is the feeling present in a room when you enter, is more than color....and can be ANY of a billion, neutral or otherwise.

You need to provide some inspo of spaces where YOU like the look and feel. Helps us to not try to read your mind: ) as to a feeling or style