Gray is a huge trend....lasting far too long. I would not use gray especially when it could very well make everything else look wrong. SW Kilim Beige and Nomadic Desert do not have undertones.

get some neutral color . I'd go with gray with blue undertone is calm color and goes well with brown, tan.

Or something sagey

Have you looked at Versatile Gray and Revere Pewter? Depending on the lighting, they are both fairly neutral and could still pair well with browns

If you just want to get rid of the yellow use Accessible Beige which is a light taupe.