So you have northeast light but plenty of it, how about going blue: this is F&B Hague blue.

But I do see your burgundy and gray drapes that wouldn’t work with this, so I get that you may want to stay neutral.

Which of your samples, or any others, matches the lightest shade of the lightest stone? Move your plaid chair and take a closeup of the top and bottom sample right next to the stone.

Warmth in the evening can come from lighting, from different art, rugs. Make sure to try out the paint on the wall closest to the fireplace during the day and at night.

Art should be more inspiring than your bear! Or no art but a lively patterned rug.

Mohawk Home Adobe Area Rug, 5'x8' · More Info

Safavieh Madison Collection MAD460 Rug, Grey/Orange, 5'3"x7'6" · More Info

Nourison Celestial 6'7 X9'7 Sunset Area Rug 099446483171 · More Info

I would sample BM Shiitake Mushroom

Pale Oak is too light for a cozy ski lodge.

Some inspiring stone fireplaces with and without art. Take note of wall paint, rugs, furnishings. Here the wall paint matches medium stones.

Elk Mountain Scenic Hwy · More Info

Modern Living Room · More Info

“Which of your samples, or any others, matches the lightest shade of the lightest stone?”

Agree.

This is the closest I can get to the stone without taking the samples of the poster board.

(I like the bear)

Since you're going to paint anyway, paint a couple of larger sections so you can view at various times of the day and see what you like best. Sherwin Williams Alabaster is another nice warm white that you may consider if the samples you have get “cold” at night.

I like the neutral palette, but also liked @lisaam idea of the F&B Hague blue, so I included this pic in case you were considering a color at all. I removed the window treatment since that didn't work with it.

I’d love to go dark but that isn’t allowed.

I like the lightest one you’ve shown

I like the top of your two samples, and also the bear. :)

I love sage green, which is warm, soothing, and natural.

I agree with Emily about painting large samples on your wall. She's absolutely right about paint colors looking different throughout the day - and at night, with your interior lighting. Many paint stores sell small cans of popular colors for that purpose, or it might be worth buying a couple of quart cans of 2 or 3 possibilities. Some folks suggest painting samples on poster board so they can be moved around the room, but I've never had good luck with the poster board sample looking anything like it looked on the wall.

Paints have to be sampled in your space, with your lighting & furniture, woodwork, fireplace, etc. Anything else is just guessing. Either that or hire a color consultant.

This house is begging for green walls. You said you can't go dark, but there are countless mid-range greens to choose from. I'd be careful not to go too light.

I love Healing Aloe BM with any wood and a true neutral. I cut it sometimes for just a hint of color but in your space the most i would cut it is 10% and let it be . IMO Pale Oak is too gray. Do not test on your walls the paint already there will have an effect on the new color. Get large poster boards , sample cans and do 2 coats then move those around at different times of day and in different spots .

Here are Pale Oak and healing aloe sXs

Sorry, ignore my rug recommendations since I see you have tall red and tan plaid drapes you may not want to replace.

I like the lightest sample since it harmonizes well with the stone, the chair, and the tan in the drapes. If you're still concerned about nighttime warmth in the room then a rug in reds and tans will work.

BM Grant Beige - a khaki with some green, that will go with most wood colors.

I used it throughout most of my previous Spanish style house.

B.M. Collingwood . a gorgeous, flexible off white.

if you can see the paint color in the stone, and that color occupies a fair amount of visual real estate, then it's an option; and the chunk of visual real estate in the stone is also is what makes repeating that neutral color in the wall color make sense.