Uh oh.......... (you will get many posts, this is a common debate here so put on your seatbelt) you will get many options to choose from and certainly will find your answer.

I wouldn't do anything on the side where the wine rack is. The backsplash serves the function to keep the wall free from splatter and so on. This area looks more like a little mail drop nook or similar. I'd place a round or rectangle tray to the left with a couple wine bottles (or small bar on it) and make it look pretty with some other little complimentary things on the tray such as below and do nothing on the back wall.

If you feel you must, then I'd stop it and then start it......

It's back splash not side splash - so just do the back, eliminates many dilemmas.

I like your first thought to finish the backsplash to where the kitchen "ends".