Once the flowers fade there is a period of ripening that happens just like with fruit. The seed will change to a dark color and the stem and caylex will turn to a buff brown. Cut them off the plant and then you should be able to get them to release from the plant easily. By the way the green part around the seed is called the caylex.

Seems my brain was working on phonics level....the correct spelling is "calyx" not caylex.

Oldherb, I really appreciate the info (regardless of spelling LOL)!