We’ve been toying with a white linen colored concrete surround. To test things out we used white cardboard. It just looks so big and with the surround.

It looks like an oversized mouse hole without a surround, especially with the unfinished baseboard gap. The concrete would be fine, if you want to keep things minimal, and then hang art above it. You could also try something in a gray, to tie in the hearth a bit. I guess you don't want a mantel? I would think that code would require something other than drywall, for sure!

Looked for a few photos that might show minimalist treatments like you propose. Most seem to involve a tiled wall, but here are a few others that might give you ideas.

Northshore Residence · More Info

Living Room with Beautiful Fireplace · More Info

Florida · More Info

Louie · More Info

Living Room · More Info

Contemporary Living Room · More Info

If you really want the firebox to stay/look small, you'll need a wider expanse of stone or tile cladding for the wall.

Central Park West, New York City · More Info

New Castle Modern · More Info

Hope this helps!

@HU - How high would the blue mantle be? The top of the firebox is already 48inch off the floor. A 8inch mantel would need to be min 8inch above the firebox. This would mean that the mantel would be at 56inch.

@ACM - the firebox is taller than it is wider. This has been our biggest dilemma. And it is approx 11inch off the floor so any surround turns out to be huge.

The hearth is wide so I would remove the baseboard, which really doesn't belong over the hearth and use whatever the hearth material is for the facing of the fireplace

Yeah, um, my first suggestion would be to go back and plan the whole thing before picking an atypically shaped firebox. but at this point, either a minimalist surround or a full wall of tile would be my suggestions.

Here's my thoughts:

This mockup features wide format tile. A Custom Steel Mantel and unusual Custom Floating Fireplace Screen. You most likely could not install a wood mantel that close to the opening, but you can with steel.

Sorry just seeing your question.
I just sketched the photo. Didn’t do any measurements.
But I’d say to take the wall and divide it by three. The mantle should be at the top of one third.
But it’s really more so about how it feels and looks to you in the space than any measurements or rules....

Hello! Really curious what you decided to do -- your before photo could have been taken in my house... :) 3x3 firebox is 14 inches above floor, flush in wall -- I would like some sort of surround, but really not wanting a hearth. Do you have an after pic?