I would definitely put a good quality primer on it before I painted. Let it dry as long as possible and you should be okay. Also use a good paint for the topcoat. I recently bought Valspar semi gloss and it's horrible, shows every brush mark and doesn't level at all. Back to Benjamin Moore for better paint.

That test is actually kind of funny to me. I mean, it's a good test, but to say that the layer needs to be removed is really extreme. It's duct tape after all. All painted surfaces are somewhat sensitive....it's why we use blue tape. If you started with a new door and primed with some really good primer, then the primer would likely be stuck better, but it's just not worth it to strip. Just sand like you did and prime with a high quality acrylic primer and then topcoat with some high quality acrylic paint. The reason you can scratch off the paint easily after painting is that it's not cured. In a week, it will be stuck better and sometimes acrylic paints can even take up to a month to cure. Curing means better adhesion so it's impossible to judge the adhesion of a paint until it is fully cured.

Gardz is supposed to be a great primer for locking down old failing surfaces prior to painting (after the usual prepping). I would think it could be used for things other than walls but am not sure. The Home Depot sells it (for much less than a paint store).

Valspar is good paint. It just needs to be applied thick and fast...and I mean, fast.