Have you thought about instead.... to paint the brick the latte colour instead of plasteirng over it? The brick would give a nice textural accent to your room.

Good point. I am wrong to put a texture on it as I don't like the texture of the brick.
Thanks for raising that point. SW

I plastered my fireplace with a very smooth finish and painted it with paint used in bathrooms so it has a gloss to it. You could use the same color as your walls or go a little darker. Mine is darker.

Depends on the feel you want to achieve. I always considered fireplaces as a feature in a room and it would be a shame to treat it the same way as the rest of the walls.
Personally, I love the texture of brick, it adds a lot of character and warmth to a room but if you don't then plaster over it and paint it a different color.
The good thing is that the rest of the room is painted in a neutral color so you have a vast array of colors to choose from depending of the look you want to achieve.
There are four ways you can go about it:
Deep/Dark colors to achieve a dramatic look (burgundy, olive green, chocolate brown, navy blue etc);
Similar shade colors like teals, lavender etc. to achieve a more subtle look but still interesting;
Paint it a couple of shades darker that the latte to give a more even look.
If you use the same color then use it as a backdrop for art to add a splash of color.
I hope this helps

P.S - You can also paint the fireplace wall lighter than the other walls if you want to make it stand out!!!

Here are some examples of what you can do. In my opinion a different color or a different shade is the best way to go.

I would only plaster it with a texture if the texture and color I was trying to achieve was a modern look of cement. Otherwise I would plaster it smooth like the rest of the walls.

When I bought this old home of mine, the bricks were plastered and painted a glossy white like mentioned in another comment. It makes the fireplace stand out. I have not changed them. Both the living room and den have the same treatment. I agree with all the combo effectc mentioned in other comments

Tone on tone using texture is always interesting.......