I've been growing castor beans for several years, and have had no problems relating to their toxicity, or invasiveness. If you're concerned that a child or pet might ingest a seed, the seed stalks are easy to remove before the seeds start popping out. And I wouldn't really call it invasive, especially compared to mimosa, or something that spreads by runners like bamboo or tetrapanax. Sure, if you don't cut off the seed stalks in the fall you'll get a few seedlings the following spring, but they'll all be within a few feet of the original plant & are easily pulled up or transplanted. Castors will NOT carpet a large area or whole yard with seedlings like a mimosa from a neighbor's yard three houses down will.

Thanks, John! Exactly what I wanted to know - no small kids in my house; one old dog who rarely chews on anything outside and then only grass. I'm going to plant them!

You'll resolve those issues if you simply remove the flower heads before the seed pods develop, as John said. Keep an eye on the plants so that you can take care of that little detail and you'll be good to go.