Technically, they go to many flowers that are not red. They are actually looking for a certain shape of flower (tube or funnel) and many of those happen to be red. One of the biggest attractants is the "Black and Blue" salvia (Salvia guaranitica "Black and Blue"), which has no red in its flowers at all but sky blue! They also like the wild japanese yellow honeysuckle (Lonicera japonica and my visitors liked my yellow/orange Mandarin honeysuckle) and they will go to many other other-than-red flowers (I had one who like my blue-purple Salvia nemerosa "May Night" and others would peck at my yellow tomato and tomatillo blossoms).

Usually the flowers themselves evolved to attract certain pollinators and the yellows tend to attract more bees and wasps (who do "see" that color as a nectar/pollen source for themselves). But because those insects can't get down into tube flowers, those types have adapted themselves for butterflies (with a long probiscus) and of course hummingbirds (with a long beak and tongue) and reward them with some nectar in exchange for pollinating them.

Jenny is correct is much that she has told you and hummers will investigate many different colors of flowers, but many flowers hold no nectar and they wont return to those. Over time the hummers have learned that more flowers that are red tend to have the nectar they seek than other flowers even though other flower color also have nectar. It appears to be a time learned effect and therefore will be attracted to anything that is red. Just as many a hummer has gotten trapped in a garage the reason being they were attracted to the red handle on the garage door pull down. Also they have learned the tubular or trumpet shaped flowers tend to have nectar regardless of color. Also the trumpet creeper vine is said to have one of the most potent nectar contents of which I dont have room for in my yard but wish I did, for It is rather aggressive covering 20-40 ft.