Hey slyedog! I have the same plant that is actually a transplant from my Grandmother's! We've moved it each time we've moved...so don't believe the hype when they say it isn't a good mover. It may stunt it for the next year or two, but they will do fine as long as you give extra attention.
The stems, leafs and seed pods are the dried items you are seeing from last year. go ahead and pull that out. Should just pull right out of the soil. If it doesn't , go ahead and cut with shears close to the ground. The new growth is the purple buds you see peeking up through the soil. The will kinda look like asparagus until they are taller.
You will love this shrub when it is in bloom.