No problem year round. My red wrigglers not only survive our relentless Maine winters, they can even be found alive under a bag of leaves on a relatively warm Jan. day. No problem with your plan. Your worms will eat the compost and poop out your fertilizer.

In my garden the worms seem to work on an "if you build it they will come" principle. I have never added worms, but the natives flock to my beds mulched with straw and used coffee grounds.

Mostly my experience has been like Richard's, i.e. make a compost pile and the compost worms show up on their own.

My current compost bin's location seems to have no native compost worms (although not far from where I lived before).

Mostly worms will show up in locations that are right for them, no extra work required. If they aren't in a specific place, it probably means that conditions aren't right for them (leaving aside there are some locales where they are not native and are not widely found). Where I am, they're common and live perfectly well through very cold winters (although not very active), they re-appear every spring. It doesn't mean they're universal though, even where conditions would seem 'right.'

So: sometimes adding compost worms to an outdoor pile does work, they just haven't physically spread to that location. I would suggest doing both, adding some to your compost/mulch beds and letting them do what they want. Keep some aside for a bin and operate as usual.

Note: there is a debate and concern about 'non-native' worms being introduced in places where they don't exist already. This might apply to islands and some other areas as well. That information is likely mostly local.