Clearly underwatered; maybe underfed. I would treat the entire tree with a copper oil spray to combat fungus, retard scales, etc. Treat it when the weather will be below 80F so as not to burn the leaves.

It looks very thirsty to me, underfed and in need of some tlc. Any fruit that is on that tree that looks overripe and not green as in this year's harvest should definitely be pulled off like you are planning to do. What kind of orange tree is it? Is there any way you can find out from the previous owners the cultivar and possibly the rootstock it is on? That will help you better care for the tree. Once you see the tree starting to look better I would fertilize it with a good citrus fertilizer, this will be the last fertilization this season, until about March if next year for the San Diego area.

Evan

I have no experience with in-ground tree, but agree with Evan that it looks very thirsty. Don't know about the brown marks - but Evan is a pro!

Thanks for the thoughts! I don't know if the previous owners know the type of orange tree. We had asked before I am not sure they knew.

Definitely Thristy! Give it a slow hose dribble for several hours and over the whole root zone. Consistent food and moisture will get the tree back to producing fruit.