I see the issue. Is the installer still on site ? Like any thing it really is subjective. I think it looks like what it is now , thatis looks like stone facing ona wall no illusion of it being an actual stone FP. Ask the installer to give you a visual with a few stones to see how it looks to add to the side walls .He did a really nice job of the face so it might be best to leave it as is but he can at least show you wat the sides will look like if done. . BTW why are your baseboards in before your flooring ?I think for me I wouldleave it since I think when the stones were done they were done so perfectly on the edge it might be a mess now but you need to have an honest chat with the installer. Is this a wood burning FP? I ask because usually you do not need a hearth with a gas one at least where I live they are not code. The reason I ask is that FP would have a bit more contemporary look without the hearth. I think it would be good to know what the plan is for the whole space too since no builtins are now the plan. I will say I do not like a bunch of cluttered shelving beside a FP unless they are needed for some special things thta need display. Are you planning on a TV on thta wall? I guess I am asking for a bit more info

I agree with Patricia Colwell, above. I also note two electrical boxes in an odd place to the right of the fireplace. Are they for a TV? What is planned for that area? Again, we need more information. What is planned if builtins are NOT in the plan?

To be honest, I’m not 100% sure. The built ins were going to be a cabinet below with shelving above. It would’ve taken up most of that space, so the side walls wouldn’t have been seen. When we decided we didn’t want the TV above the fireplace covering the stone that we just paid money to have put up, we decided we would put the TV to the side, hence the high up, oddly placed outlets. When we decided to put the TV to the side, that is when we decided to take the built-ins out. Wanted to get more of a visual of how things were going to look before we committed to putting anything else in the space. Figured it’s something we could put in later. I do really like the look of shelves, no cabinet (picture below). But I feel that will still show a lot of the sidewall. For the side where the TV is not going the other option is possibly to just leave it open and utilize that area for holiday decor like the Christmas tree or a small piece of furniture like a chair. I guess my biggest concern is that if we don’t put built-ins or anything really covering that wall by leaving the front only with stone, does it make it look cheap and unfinished.

Stone like this would require corner stone that wraps around the sides. If you put stone on the sides now you will see a line of stone on either side from the front-a.k.a UGLY

Leave it and maybe paint the side walls a dark colour that is in the stone so it does not look so prominent.

I see what you're saying. The stone veneer is obvious from the side. Install some sort wood paneling on the sides so the fireplace design looks cohesive and intentional. Match the wood to the mantle. I'm thinking vertical planks. That mantle is beautiful wood.

Do the built-ins with just the shelving that you like. A built-in would have a wood side to cover the side of the fireplace.

This is also a good reason for hiring an interior designer if it's in your budget. To help you visual and help you make choices. :)

I would do the built-ins without question. Not only to solve the issue of the fireplace stone veneer with unfinished sides, but also because that fireplace looks lonely and isolated with the empty spaces on either sides. And it's nice to have a place for books, photos, and the like in the living room.

Here's the TV solution: you design the built-ins to have the TV on one of the sides. Install a TV mount with articulating arm. I recommend highly the Peerless SA761PU. The articulating arm enables you to pull the TV forward of the built in as well as tilt and turn it how you want for best viewing. When not watching TV, the articulating arm folds the TV back into the built-in for a clean look.

https://www.peerless-av.com/products/sa761pu

Ask the installer how he would do it if he wrapped the sides.
If he will remove some of the end stones and redo it, properly, I think that’s what I’d do.

I also like the idea of the shelving and or built ins.

I would also see about taking the stone all the way to the ceiling instead of having the white crown molding above it.