I live in MN ( currently 3 F) and I have no problems, the bottoms on mine are slate though. The bottom will be weak the point if there is any condensation or from rain.

"Condensation / Frost" is caused when moist air comes in contact with a cold surface. Talk to your window manufacturer and your HVAC contractor about the potential condensation / frost that would accumulate on the interior of your windows. Also let your architect and builder know of your concerns.

I wanted a clean look without drywall. I used MDF returns as in this article. I used water resistant MDF. So far so good in chilly upstate NY. https://www.finehomebuilding.com/2020/01/03/mdf-window-returns

We get plenty of cold for a long time here in AK. The house I built 3 0 years ago has drywall in the window returns as do many others. If the windows are installed correctly and sealed from air leaks you should have no problems. I would suggest caulking around the window frame to the drywall also. If you want you could install a wooden ledge board on the bottom sill. Because we deal with minus 4 0 temperatures, I install the shrinking plastic around the windows also. Really makes a big difference. Good curtains will also help