Love a challenge!
Do the wooden shelving units NEED to stay?? Because I would say good bye to them....then the room would be en up vs...cubby fireplace. Are you painting the fireplace? Distressing? Darken the mantle! A hunk of barn board would look fabulous.....do tell

It's very interesting! Biggest flaw is the rug color. It isn't working with anything else.

If the hearth could be modified to go straight across perhaps there is room for two club chairs and an ottoman.

and if possible take the shelving unit away,then decide on either a new carpet or a wooden floor...hickory..bamboo..oak.....what is the style of furniture you have or like?hard to tell from the empty room...

and can you tell me what is behind those wooden walls next to the fireplace??

Thank you everyone for your thoughts!! We really appreciate it. We are note sure if we should paint the fireplace, but I think we are definitely either going to paint the paneling or get rid of it entirely. As far as we can tell, there is drywall behind the paneling.

As usual, houzz has got you covered! See ideas here:
https://www.houzz.com/magazine/inglenooks-carry-on-a-warm-tradition-stsetivw-vs~22970376