The only way to eliminate scale is to treat for it - there is no natural control or predatory insect that will completely address the issue (although they can help). And the recommended method is with a horticultural oil applied at the appropriate timing of the life cycle of the scale organisms. Ants can be more easily deterred if you wrap the trunk in a sticky tape or something like Tanglefoot to keep them from climbing the tree. They are just after the honeydew or the sweet, sticky substance the scale excrete - if you control the scale you will also be controlling the ants.

OK, then. Tanglefoot at the base and insecticidal soap for scale. I think the jury is out in terms of using neem oil on edibles ... is it safe?

We'll have to get up there with a ladder. I absolutely hate the risk involved in that!

Neem oil is perfectly safe to use on edible plants. Clarified hydrophobic neem oil (without azadirachtin) is made of fatty acids and glycerides, which are substances found commonly in many food products. Azadirachtin can be irritating to the stomach but prepared neem oil generally carries very low concentrations of this substance (if any). And neem oil is frequently used in many cosmetics and personal hygiene products, including various dental care products.