These look like flower stalks, not pups.

I agree with Rina, they look like flower stalks. No need to cut those :)

Yeah! So glad because I think they're pretty and interesting. Will they eventually die off? I assume I cut them once the stalk is dead.

You can leave them on the plant until stalk start drying up. Or if looking for 'perfectly looking plant :), cut off as soon as flowers are finished. Your choice.