A close up would help, but my guess would be to take a color from the brick (or is that stone? )

It's yellow-ish, orange-ish, brown-ish brick.

The white trim is reading gray/blue on my monitor. Could you possibly post a larger photo, or a close up?

Dark gray

Deep charcoal trim and windows, downspouts painted to match brick:

Another vote for a charcoal grey!

I like the charcoal gray too, however, cant see what roof color is, to know if gray will look good with it.

Paint them if you don't have the budget to replace them with glass. Match paint to darkest color in your brick &/or roof.

The paint colors should be reversed on your house. Your windows and trim should be the darkest color pulled from the brick a very dark charcoal brown while the panels should be the lighter color within that dark color span. Because I don’t have a straight on shot of you house, it is very difficult to paint your house in a way that would show you what I am talking about via the computer. Here is another house - that had a similar issue. Hopefully you will understand what I am saying.