I don't know of any varieties of rosemary that can survive zone 4. Barely can get them to here in 5b, and that's only in microclimates. Grocery store rosemary is generally Tuscan and only hardy to 7 or 8.

When repotting my rosemary, I do cut about a half inch of the bottom roots off, and snip up the sides in about 4-5 places to help spread the roots. A big pot is not better for rosemary, as it can lead to too moist soil and root rot. You want to go no bigger than 2" more than your rootball on each sized. Indoor rosemary is prone to powdery mildew too if airflow is not good.

Good luck, but don't feel terrible if it doesn't make it. Grocery store plants are not the healthiest--they're generally forced for leaf growth, since they're intended to be used all at once more or less.

These plants do really seem forced... why else would anyone jam three plants into such a tiny temporary pot? I've had good luck with grocery store herbs in general (minus the mint that got an infection after pruning and died a slow, painful death) but rosemary from all sources including nurseries seems to hate me. I'm thinking it's an issue of not enough light and am thinking of keeping a fluorescent light near these.

Rosemary in winter is very difficult regardless of its variety or grocery store/nursery source. I can't overwinter it outside. My zone 6, even in microclimates, kills it every time. Overwintering indoors isn't much better. I suspect the dryness of our winter heated houses is the cause because the only time my Rosemary made it to spring it was in the kitchen window above the sink. Good luck!

FataMorgana

My little sprig of rosemary I started in water is alive and well in my master bathroom with an east exposure. I potted it up into a 4" pot. I mist it daily and water it about once every week and a half.

And does that pot have a drainage hole? It looks more like a cache pot. Rosemary must have sharp drainage.

It does have some drainage, but the four holes in the bottom are pretty small so I don't know if it's enough. This is a very temporary pot since it's too small to even fit the entire root ball. :/ Part of the reason I posted is for pot advice.

I do manage to overwinter my rosemary. I mix some bonsai type soil in with the potting mix for extra drainage, and don't get mixes that have water retention beads or similar added. If you go with a plastic or similar pot, you'll need to be extra careful of the watering because the pot won't absorb extra. A terra cotta or unglazed stoneware pot will absorb some water, so it will need watering more often. I water only when I see some signs of drooping tips, or I also use a water meter to keep track too. Misting with spray can help, as it prefers more humidity than I have here in Denver. I have my larger bush by a back door with East exposure and it even bloomed for me this year. The smaller babies I keep under lights grow more due to the exposure. All are happier when they can go outside for summer. But it is finicky, and by the time you notice a problem, it's often too late to do more than watch and mourn.

I've killed every attempt to grow rosemary from seed or from the nursary unless I just basically stick it outside and leave it alone. So easy to overwater. This year I hope to plant it in ground and see if it does any better because my container skills and rosemary do not mix.