Ants like to farm on the plants they wont hurt the plants directly. They also like the extra floral nectaries along the leaves. I would guess they might be useing your plants to raise their livestock like aphids, scale and mealies. You may have to use a systemic to get rid of the pests then the ants will find other fertile grounds to graze thier stock on.

I am having the same problem my impatiens are starting to die.
I've noticed clusters of dirt all around the buds at the top of the impatiens stem. When I pick it off and pull the dirt/bud/flower cluster apart, there are tons of ants congregating inside the cluster.
I thought perhaps this was due to stink bugs biting in to the plants, leaching sweet substances that may be attracting an unusual amount of ants. At the rate this is happening, I will lose all of my impatiens.

Let’s wake up this sleeping giant. How do I keep ants out of my 2nd floor atrium? I have a garden there.