Understandable. Take the most neutral gray tone out of the stone for the siding color, and darken it for the trim.

Grays are always going to read blue. That is why the little bit of gray stone on your house looks blue. Consider a warm gray tone as your trim for the major timber elements and do a lighter siding tone to echo the warmer stones .. test http://www.benjaminmoore.com/en-us/paint-color/cromwellgray

maybe something light but in the direction of the stone for the siding and garage door - like bm bleeker beige http://www.benjaminmoore.com/en-us/paint-color/bleekerbeige - then the house will look bigger because it will all flow .