I don't know what you're envisioning when you say brown. It sounds dismal to me, but if you have looked at siding and you have a particular color in mind, please post a picture of the sample. We could also use a better picture of the brick. The one picture you posted that shows it is in shade.

Based on what I see, I would do a cream (light color that dials down the yellow in the current color) or taupe that complements the brick. There might be some greens that would work too if you want more of a color. If you have chosen a siding manufacturer, post a picture of their sample board so we can see what you're choosing from.

Your garage doors are a large part of the front facade, so although they're not included in the current plan, think about replacing them in the future with doors that have windows and a color other than white. That would improve the appearance of the house. The garage door color would depend on which color you choose for the siding.

I found tghis color scheme to pair with your roof and brick colors. I found color matches with Benjamin Moore paints.

Green for the siding,cream for the trim, windows and possibly the garage doors, brown for the gutters, and rust for the front door.

Here’s a closeup of the brick. And the samples I’m working with are also pictured, with one possible choice.

My leaning toward brown is prompted by a sticky resin that falls from my neighbor’s cottonwood trees and sticks to my house …and everything. I have begged her to thin them out and so far nothing. Also two of my neighbors have taupe houses and I don’t really want to add another. Garage doors will be replaced in this process. I would really like a wood-look door so I’m trying to incorporate a color scheme that will allow it.

So grateful for ALL your input!!

I would not choose brown for your main house color. Keep it lighter. @ffpalms offered you a great color scheme.