Well, if you want to continue to be able to walk around the island, you won't be able to build storage on the whole wall, but you could add a line of high shelves from the corner cabinet to the doorway (or even across the doorway - I can't see what is beyond to know if it fits in) and then if there is room add a tall cabinet from floor to upper cabinet just on the kitchen side of the door - I say "if there is room" because you will have to allow at least 32 inches from the corner of the island to the shelves so you can walk around them. I would then decorate the remaining wall space - paint it a contrasting color or add a variety of artwork or even hang cookware or plates. That bare space will need something to make it pop and say "I did this on purpose" instead of "there wasn't enough room to add cabinets."

Would it be possible for the island to be a penninsula?

i would have the island coming off the corner pushing it right against the wall but first i would cover that wall with faux brick or what ever stone appeals to you and then i would take out that microwave cupboard and make that a whole wall of cupboard with uppers and lowers and if you wish to capitalize on the stone have the uppers with no doors or make open shelving whatever works for your lifestyle or design....enjoy

Maybe you can steal some space from the room on the other side of the wall. Build recessed cabinets or open the wall and reframe the areas between the studs to create extra storage.
Like this photo above the tub.

Maybe you could make a wall of plates, trays,pans,kitchen cooking utensils or spices. Here is a little something i did that might give you an idea. I used the ikea picture ledges to place trays or plates along with hung pans. Maybe you could also paint (or chalk paint-i see you have a little one :0)) that wall an interesting design to makes it feel like part of the kitchen plan.

I like the plates idea, something shallow like that could work. Also a spice rack can be put there.
Another thought would be to take your existing kitchen cabinets to ceiling height, adding more closed storage.