Our shutters have a little navy in them (they are not black)

Gutters and downspouts: same color as the trim so they don’t jump out. (Look at meeeeee!!! I’m a gutter!!!)

Agree, I would want the trim and gutters same color so they'd look as one. For the downspouts, I've always used the color of the house itself so they 'disappear'. I've never had brick so am not sure how I'd handle that, do you have a clever painter or clerk in a favorite paint store who could mix to best match your brick.

Gutters should match the trim, the downspouts the brick.