I would remove that tall cabinet where you wanted to put the fridge. You now have a blank slate in that area to put a larger fridge ( you can never have too large a fridge). Then you could add a wall to the right side of the fridge to house and frame your fridge and add some counters and cabinets to the left of the fridge. Or, you could add some floor to ceiling cabinets to the left of the fridge ( you can never have too much storage in a kitchen).
I attached some pictures of my renovated kitchen, to show how counters and cabinets could look with my suggestion. Your kitchen is lovely!

I would put the fridge to the right of the cabinet next to the dishwasher. I can’t tell if there is enough room, but you indicated that you were looking at a smaller sized fridge. The fridge in that location gives a nice ice-water-stone-fire flow. It will also be directly next to the common rooms so it makes it easy if someone wants to get a drink without getting in the way of the person cooking. And, yes, frame in the fridge if you have the resources.

At the point you want to do some updates, move the vent and add a wall of shallow cabinets to the other side so that the new cabinet run ends in line with the fridge.

I would put a counter depth fridge and cabinet configured like this so you can have some landing space next to it. Depending on your budget, you could also face the fridge with cabinets so it disappears
.

All of the examples posted have cabinets over the fridge--but you have a vent there, and tall cabinets, or standard uppers, might make the area above the fridge look like a cave. I know it will decrease storage space, but how do you feel about open shelves? I'd try to find a tall fridge. (kculbers, I hope you don't mind that I used your pic).

If you must keep a tall pantry, maybe use a side-loading cabinet to the leftwith counter to the right: