Clip them short before you dig them up. I usually take a hand clipper and a paper bag and clip it and put the pieces in the trash.
You might want to see if you can list them on freecycle in your area. Someone may want them and even dig them out for you.
They attract birds and have lovely berries in the winter. In the right location they can be a real asset to a garden.
Quite possibly the previous owner liked the picture of the plant on the tag and never realized how thorny they are.

I never dug one up but suspect it is not too hard once they are pruned to the stump.
Cherry

IF you can find a local bonsai club, they'll run-not-walk to dig them for you, and will know how.