astilbe are generally hardy. but you probably should plant it in the yard later this summer/early fall. they are good companion plants with hosta.

It is unlikely, IMO, that the Astilbe will survive the winter. You'll likely be hard pressed to find plants that will. (Though you might give a mini hosta a try.) A window box will not receive the full benefits of snow cover and the moderating effect on soil temps being in or even in contact with the Earth provides.