aspen, I leave soil in my lightweight foam pots and even leave some perennials or cold loving plants [such as mums, asters, etc.] in them. The composite material of these pots also seems to insulate the plants to a greater extent than clay would. I've not had any crack or show damage yet.

Of course, your zone is different. I'd think it would be possible if you placed them in a semi protected location.

What you want to avoid is the freeze-thaw cycle - the expansion (freeze) will crack the clay. (Thus my forgotten but bone dry clay pot came through last winter fine.) I will be planting some early flowering bulbs in my pots this winter - as a balconeer, I have an aversion to having plant-less pots of soil taking up space. The perennials will just stay in their pots.

I put black plastic trash bags over my large clay pots - even with soil in them they seem to come through the winter fine. They look ugly but they survive.