the colony will survive, even long freezes, the worms will leave cocoons to hatch out when it warms enough again, just depends on how much you want to fuss and what your situation is

keep a bin inside .... keep it in a garage or outbuilding that doesn't freeze .... bury it part way to take advantage of ground that doesn't freeze, and cover it with an insulating layer .... put up with less activity in the coldest times

Bill