color -wise, the colonials weren't just white.take a look at some of the historic Williamsburg colors used for siding - yellows, greens, tans, etc.
that being said, with the nice brick veneer, possibly a warm tan or gray? - keep the white trim, and change your shutter color to match your front door color, both complementary colors to the siding. pull up Hartland siding - one of the best my builders use, if you're going to stay with vinyl. if you're residing to a cement board product (like Hardplank) then the world of colors are available.
on my reside, i went with a putty color siding, and the shutters are a dark dark blue with an lighter shade of blue for the front door. my brick is about the same shade of yours, in the warmer tones, too. good luck

I don't think you should limit yourself to thinking only colonial for this house. This pairing of siding and brick isn't really common in colonial architecture. I think even if you get the most perfect colonial colour scheme, it doesn't ever look quite right on this style of house. If you're open to something less traditional, you might consider a natural wood siding instead of vinyl. The wood in the picture is ipe and it is about three times as expensive as standard quality vinyl, but other types of wood like garapa are only about twice as expensive and both types are around the same price as high quality vinyl.
If you are set on something more traditional, I'd go with a neutral grey and white trim and shutters, although it might look better without any shutters.

thethistle, I love the way the wood looks, BUT, would it look terrible on just the front?

I would look at colors that go with the brick. A grey or light tan would look good.

Good luck:)