any closer pics of the leaves? are their caterpillars eating it?

I can't find any evidence of caterpillars. It looks like something is biting off branches. I'll post some additional pictures. Thanks for the response.

Here are some more pictures

Caterpillars can be pretty difficult to see. There's one type of caterpillar that eats citrus trees and it looks like a curled up dead leaf. You can spray something with Bacillus thuringiensis onto the leaves and see if the desctruction stops. I know it works really well for hornworms.

looks like you're getting a lot of new leaves flushing out, which is a great sign that it's healthy.

iandyaz is correct, the Orange dog caterpillar will eat citrus leaves, especially new leaves. check around the plant and underneath the leaves for them and kill them. Spraying BT on the leaves works too.

Do small wild animals roam around your neighborhood? It could be rabbits or javelinas?