Do you have bee guards on the feeder?

Does the feeder get direct sun? This often causes the feeder to leak, which is probably what is attracting the bees.

Why does it matter?The hummingbirds can still use the feeder, bees aren't likely to sting anyone who doesn't really bother them, and they probably wont eat much of your nectar either. Unless you are deathly allergic to bees you should be able to simply reach up and grab the feeder with little apprehension.

I'd advise you to keep an eye on the feeder, bees don't swarm it all the time, find a time of day when there are fewer bees, early or late in the day. You should be able to take it down with no problem, move slowly, and don't wear blue color clothes, it's bees favorite color and will spend some time investigating you to see if you have pollen they can harvest. Good luck.

Chances are it is yellow jackets that are around the feeder, if so, put a piece of meat a few yards away from the feeder and the yellow jackets will go for that too, and you will know that they are yellow jackets and not honey bees (which are not attracted to meat). Yellow jackets are more aggressive than honey bees and should not be messed with during daylite hours.
Wait til dark and take the feeder down.
If they are honey bees take the feeder down (if you are afraid of bees, again do this at night) and leave it inside for a few days. Honey bees will discover their nectar source is gone and move on. they have short memories and will forget it was ever there until a scout finds it again. I doubt it is honeybees because the feeder is not what they would consider a good source of nectar unless there is nothing else available. I have hives and feeders and the only tijme I have any problems with bees at the feeder is with yellow jackets, which sends me around the yard in search of their nest, so I can destroy it.

My hummingbird feeder has been swarmed by honeybees most of the summer. Not one or two bees, but 20 or 30. I am looking at Amazon today to buy a bee proof feeder. When they begin to swarm, I squirt the feeder with a hose and while the bees are gone, take it down. I’ve been fighting this for at least a month. Time to try something new.