Bottom baseboard and crwon removed? All the way on the right side? Thanks :)

I'd consider stacked ledgestone instead of brick with this contemporary FP. Search the box above for "ledgestone fireplace" under dropdown "photos." You won't believe how many people on this site are ditching brick or looking for ways to cover it up.

I said brick but it is more of the stacked ledgestone like you said.

Be careful of these panels, most of them that I have seen end up looking like puzzle pieces, because the perimeter pattern repeats. A true stacked ledge stone never repeats, this would be something to consider for optimal results.

HI -- I think if you added a board ( vertical) down each edge of the wall ceiling to floor - one beside the stairs and one on the right edge of the wall then that would give you an edge for the stone panels to finish off. Now this will leave you with a little triangle of wall on the bottom left side that you should just paint in the white trim color . Paint the side boards the same white. This will give you a nice frame around your stone work.

This is not really addressing your initial question, but another option is to simply put a frame around the fireplace so you don't have to worry about where the stone begins and ends.

You have several issues, that in my opinion should have been addressed before now. This is cool looking, but you have what appears to be a vent, there is an outlet and there is crown molding. All of those issues stand in the way of going floor to ceiling--which is what I would recommend. The other issue is thickness, if you will...Tile has no thickness to is, and it may look as if it is just stuck on there. Ledge stone may be cool, but again there are issues. Also, and I do not mean to criticize, but the fireplace looks too high in the wall. Floor to ceiling and equidistant on left and right is the answer you seek. Good luck.

The vent was put in my mistake it will simply be covered. It is electrical fireplace so we need outlet. We wanted contemporary look so hight of fireplace is at eye level shen sitting which is what we wanted.

Crown was put in by contractor but will be removed when we build wall for fireplace. The FP is inserted into the brick wall.