Ants are attracted to the sugar. Are you cleaning the outside of feeder well. You might try cleaning the outside of feeder with ammonia water. Does the feeder leak? or move the feeder location.

Get an ant moat that hangs above the feeder that you fill with plain water. That will put an end to your ant problem because they don't like the water. I have also been told that bay leaves hung above the feeder will also repel the ants. I am not sure about that since I haven't tried it myself yet (I just bought some bay leaves today) but the ant moat definitely works. I have been using ant moats for 5 yrs. with 100% success as long as you keep water in it.

Penny

You can put petrolium jelly on the pole and or feeder ports so the ants can't get access to the nectar.

Petroleum jelly is frowned upon by hummer enthusiasts in particular banders and researchers. If a hummer or any bird got it on their feathers from brushing up against it it would not be good. Water or the bay leaves are both completely safe.
Penny

I'm with Penny - Get an ant moat! The hummers hate ants in their feeders and will not drink nectar from a feeder with ants swarming on it. Before I hung my ant moat up, the hummies were screaming and flying around their feeder, but would not drink from it. I had to see what was going on and when I looked, saw a whole lotta ants in there.

As soon as I cleaned the feeders and hung the ant moat up, everything went back to normal and my birds were happy again.

I like the Parasol Ant moats. I put the petroleum jelly deep inside where the birds will not come into contact with it. Or you can fill with water.

I get them from this site. I like them because they charge only $3 bucks for shipping & handling (US Mail) if you order accessories or parts and buy less than $17 worth of items. (I think the ant moats are around $7).

Here is a link that might be useful: Hummingbird feeders, parts and accessories