Amritab, you wrote: "I might have watered it too frequently (every 2 weeks) and let it dry out a lot before the last watering."
I don't understand the problem..A plant should be well-watered, then allowed to dry before adding more water..
I allow my SOJ to dry a lot in winter, doing so promotes fragrant white flowers during winter months..
BTW, what size is its pot, and how tall is your plant?
If it's overpotted, and you watered every 2 wks, used heavy soil, it's possible the soil didn't dry properly..(top might have dried out but mid to bottom remained wet) Could this be the case?
SOJ's cannot live in muddy soil.
Remove yellow leaves..also, while you're at it, inspect for insects, especially scale. Brown bumps that can be scraped off..to be exact, while inspecting look for mites and mealy, too. Toni

amritab, generally, if brown, crispy lower leaves fall, the problem might be too little moisture and if green, healthy looking leaves fall, the problem could be too much moisture. Could it be that you let the plant dry out too much, hence the dropping brown leaves? Then you may have overcompensated at the next watering, hence the dropping green leaves? If pests are not a contributing factor, the plant should settle down in a short time.