If you move them, cut down the bloom stalk first so the plant can readjust to its new location. Do it on a cloudy/rainy day. This works best with baby plants, which don't have a big taproot yet.

From my experience, even a tiny bit of root left in the ground will continue to grow again, so be vigilant about pulling it out till it loses its energy.