I prefer to wait until spring to cut things down. Leaving the foliage up helps protect many things from winter-kill. Anything that was diseased or is prone to fungal diseases is better cut down and cleaned up - peonies and monarda for example (peonies because of botrytis and Monarda because of powdery mildew). Some things, daisies for example, produce basal rosettes of new foliage so those can be cut down to the rosette. Experiment to see what works for you but the colder the climate you're in, the more protoction things might need. Do you mulch?

I use this rule of thumb: if it is a "Woody" perennial, then you'd best wait til spring (tough stemmed, with hollow core). If the stems are totally dry and fall away easily, then by all means prune the dead stuff away now.
Just make sure you know where you have stuff planted so you don't end up digging in a wrong area next spring!!!

The term "woody perennial" refers to one that actually forms "wood", rather than dying to the ground each year, as herbaceous perennials do. To explain further, the stems remain alive, the same as the branches of a hardy tree or shrub do. (It has nothing to do with whether the stems are tough or hollow.)
The reason one would wait until spring to cut back woody perennials is to avoid unnecessarily cutting off any stems that may otherwise have survived the winter. In spring, it becomes obvious after a certain time which stems suffered some winterkill, and which didn't, and then the dead portions can be pruned off.

You could leave seed pods on for the winter birds.