It's better than nothing. Don't let it go to seed unless you want more of the grass next year. If you are not worried about the existing annual plants that you are planning to pull out making seed you could just leave them be. Even when dead their roots will help a little bit with the erosion.

Are you planning to mow it? If you're not going to mow it and you want a more natural look, then I think some of the native grasses are more attractive when left unmown. If you don't want to mow and you want a more cultured look , I would suggest some of the no-mow fescues. Otherwise, I think the rye would work. No matter which way you go, make sure you cover the seed with something to help keep it from washing down the hill. Straw might work, I used burlap and cheesecloth since I had some on hand. I raised it up a few inches so it wouldn't pull the seedlings out when I removed it.