Depends a lot on what the weeds are. You should be able to use a grass killer without harming the cactus--something such as Grass-B-Gone. Broadleaf weed killers may not kill the cactus, but will certainly make them sick, and anything sprayed with glyphosate may take them out completely.

If the weeds are growing above the cactus, you could use a cotton glove wet (but not dripping) with full strength glyphosate and gently pull the upper parts of the weeds through your hand so that you get the weed killer on the weeds but not the cactus. This will safely remove the weeds. Be sure to wear a rubber or plastic glove under the cotton glove.

Safer than a cotton glove -- a definite no-no -- would be a paint brush to help you "weed."