Do you have a close up photo of the brick? Usually old brick has some gray tones in it, and it might work to pick up on one of them for the walls. OTOH, the gray accents in the brick might be too dark for the room.

Thats my fear @feeny - I am worried that grey (despite loving it in theory) will clash with the brick and could be too dark for the space.

Im sure there is a light warmer gray color that would work very well

Yes, I think a pale warm gray (i.e. like a light taupe--a gray with brown rather than blue undertones) might work nicely. Alternately, you could use an off-white shade (like the color of the grout, or some other off-white or ivory with brown rather than yellow undertones). Brick and off-white walls can look great together, though it would need to be enough of a shading contrast with the white trim to look intentional, unless you wanted to go with a more modern white with white trim all over.

I do not think the buttercup yellow looks bad with the brick, at least according to my computer. Since you want the sofa in bisque and more of the walls in grey, you may be heading toward too many neutrals in the color scheme. I would suggest you consider covering the yellow walls with a very pale aqua that will go with the orange tones in the brick and will provide a pastel background for the bisque sofa. Shades of aqua would look great in throw pillows and bolster pillows for the red chair and the dark brown chair too, and these little accent items would help tie the color scheme together.