Yes, the foliage looks prettier without it. You can cut it off right at the base of the rhizome, since that rhizome will not bloom again it needs no foliage.

hosenemesis thanks. Should I just cut off the stalk or cut the whole thing off including the leaves too. I think I hear you saying to cut everything off, but won't leaving the leaves help to generate next years new rhizomes? Thanks. Ed

If there are already new green plants growing off of the rhizome just cut the stalk, leaves and all, off of the mother rhizome that has already bloomed. If there are no new plants growing off of it yet, I would leave the leaves on to see if new plants sprout. The problem is that when you cut the top of the stalk off water can get inside and rot the rhizome. I have never had this happen, but other people report that it happens in their climate.
Renee

Thanks good suggestion. I think I'm more likely to lose the rhizome to rot due to slugs gnawing at it from ground level, than water entering the stalk, but I'll keep an eye on it. Thanks again. Ed.

How to remove the stalk nice and clean.
Get your hand right around the bottom of the stalk now bend it to the ground.Did it snap off? If not bent it the opposite way.This is how the pros do it.
Bill.