Are you using a little amount of zigzag? Sometimes one of the fabrics stretches while sewing and the other piece sort of stays put. Also be sure you pull on both pieces just a little bit while sewing. Good luck

You're not doing anything 'wrong'. It's the nature of the beast. The feed dogs keep grabbing the bottom fabric and pull it along. The pressure foot keeps pushing against the top fabric. If the two have different characteristics they will behave differently. If you do alot of sewing like this, a walking foot will help. They're not cheap. You can also put a strip of paper between the feed dogs and the fabric. I used to use paper toweling. Also shorten your stitch length. That will make the paper easy to take off when done.

If your machine has the feature that loosens up the presser foot pressure, loosen that up a bit. It does help keep one end from stretching. You also don't mention that you pin at the beginning, middle and end.

I just finished a corduroy jumper that started to do that until I lightened up the presser foot pressure a bit and lengthened the stitch a bit.

Also watch how you are feeding the fabric in and don't do taught sewing on knits, it stretches everything out of whack--ball point pins help tremendously when pinning. Just steer the material through and let the machine do it's thing.

with knits and stretchy fabrics, should I pin parallel to the presser foot or horizontally... or is it just personal preference?