I would remove the builtins or do the exact smae on the other side . as for the fix of the FP the builins are removeable and you will need to strip the whole area down to see what you are dealing with. If brick is the plan then the wall around the FP will need to be built to hold brick . IMO you are probably better off getting a pro to help with this than to tackle it as a DIY . I would just get a much nicer surround keep the marble and remove the bedboard and drywall then hang a great piece of art above the new mantel and remove the builins all of which you might be able to DIY.

I agree with the person above if its within your budget. If not from a design standpoint, I'd either remove the bead board and ship lap it. Or trying painting it all white. Put in a mantle and a large painting over the mantel. Then I'd take out the tile which dates the fireplace to the 80's and redo it in a light cream color. Then omit 3/4 of the books and decorate the shelves with glass accessories.