I would "hold your horses" until danger of frost is past. I am in zone 5 also, and won't plant in the ground until the end of next month, since we usually have frost then. It has been a strange spring though, so it is anyone's guess.

I have planted a whole lot of things in the ground so far over the past 2 weeks. Earlier this week, I planted some ornamental grasses, one of them being a 9ft variety.

Things you can plant now:

Generally the following plants CAN be planted NOW:

All Nursery stock including:
Trees
Flowering Shrubs
Evergreens
Roses
Vines

Perennials including:
Grasses
Groundcovers

Many vegetables like onions, cabbage and lettuce (but not tomatoes and peppers)

Many herbs

Annuals like pansies

I am in Zone 5b. My giant colocasia and alocasia tubers are doing well in pots though