The worms won't care, but the plants that you use the vermicompost on will. Weed seeds are not killed in a worm bin. Though many (most?) microbial pathogens are killed in a worm bin, there is no guarantee that plant diseases will not survive in the bin.

I would also be concerned about the roots surviving and sprouting in the bin.

I've had things sprout in the bin (corn, tomatoe, etc.) and just pull up the sprouts, tear them up a bit and toss them back in. Makes a little bit of bin maintenance for you but I'd rather kill a few sprouts than worry about stuff in the bin having seeds. Depends I guess how soon you'll be using your compost, where you're going to use it and how concerned you are of the seeds sprouting. The plant diseases would be my biggest concern. I'd rather that stuff be put into into the fire pit at the cabin than in the compost.