I would leave them until I had evidence that they were doing anything harmful. Where I live, the ants are usually busy collecting seeds and/or insects, and seem to actually benefit the soil by nesting in it. Not all ants are alike however. I would advise identifying the species and learning about them before deciding their fate.
I use cottonseed meal as part of an organic fertilizer mix. After I scratch it into the soil surface, it's quite comical to watch the ants come over and carry away little bits of cottonseed. I figure they take it underground, eat it and poop it out. Job done.

Thanks for your help. What I thought was apple gourds - I,m know not sure. I planted my garden late and the only seeds available were from a friends last year crop. What I thought were apple gourds are growing long and currently looks like a cucumber. My first garden is becoming very interesting.

I have enjoyed this site so much and check daily for new post to read.