Maybe you should consider placing the garage at a 45 degree angle off the house if building codes will allow it. The degrees of the angle might vary or be restricted by codes.

Clopay Gallery Collection Garage Doors · More Info

I'm worried that with such a tight lot, it would look strange. Also, I still think I would have to reshape the yard in relation to the driveway.

Just consider that you will lose all the windows on that side of those rooms...

My only concern would be whether or not you can get that close to the property line. In my town you can get within 2-3 ft of the lot line if it's a detached structure, but you have to be 10 ft off if it's attached. Otherwise, shifting the driveway and all that is really no big deal, assuming there's not a big chunk of stone in your yard similar to what's on the hill behind you. Are you in New England? We dealt with that a lot growing up.

^What he said. First thing is to investigate required property line set-backs so you know if this is legally possible.

Exactly and then get an architect and a landscaper to address the mess when you cut out that side of the yard like matching the old retaining wall . You need some pro help with this and as for cost I have no idea that is something that needs to be discussed with the pros you hire.

Yes I live in the rocky hilly crooked and randomly laid out Boston metro region which is a challenge compared to the soft-soiled grids of my previous home - the Chicago suburbs, where every parcel is rectangular and flat!

That said, I've really grown to love my view of the hills behind my house. You don't see this in Illinois.