The caterpillar is the larva of a Giant Swallowtail butterfly. The absolute best way to get rid of them on your tiny tree is to pick them off and dispose of them. There is absolutely zero reason to grab a pestide of any kind for this situation.

That will take an inspection every once in awhile, can you do that? They don't bite or sting and can be handled without worry.

Have you sprayed anything on or near your tree? And have your examined the curled leaves to see if there are signs of aphids or other tiny insects living in there?

Thanks for the reply rhizo_1 , I definitely can inspect once every week and cut and dispose the leaves that has larvae on, removed two of those this morning. I have used neem oil spray 2-3 times last year and couple times this year, does not seem to help though.

I looked closely on the leaves and no sign of aphids or other tiny insects that I can see. I took another picture to focus on damaged leaves. Mostly the damage is the white greyish web formation on front and back side of leaves.

Look up citrus leaf miner

Is 'diazinon' legal where you are from?