You might need to begin where the back crotch begins to curve toward the inseam and make the seam deeper, curving back into the original seam at the inseam. Sometimes we sag in this area causing us to need this alteration.

You may want to try taking in the back inner leg seam a bit. Start the take-in about 4 inches below the crotch line. Leave the front leg the same. Sometimes just taking in a half inch can do it...good luck - I have struggled for a couple of years trying to get my pattern correct - I have learned a LOT of tricks along the way. If you can get your hands on the Palmer Pletsch book " Pants for Every Body" it is a GREAT resource!