I would bring back the colonial details. So yes to Hardy plank. No to running the siding vertical at the side eaves. If, possible I would not side the scallop at the roof line but paint it white so that it stands out and ties in the columns. Painting the siding and garage to match the brick will help with it looking squatting. I’d keep the shutters but they need to be wider-half the width of the window. Add shutter hardware and go with panel style. No shutters on the center window. Add more trim to the windows if possible. Then the shutters would not have to be so long. New garage door with windows and carriage hardware. Paint front door and shutters black to tie in roof.

^I like the above suggestions from Nancy except for the front door style. Go with a colonial type door, without side lites, rather than the typical Victorian oval. You can even go a little bit modern on the door with the narrow panels.