It is highly unlikely that you have any queen traveling around since the duty of a queen ant, bee, wasp, is to lay eggs and the workers stay around her to feed and otherwise care for her, she simply does not move around except once a year, maybe, with a swarm.
To deter the ants trying to gain access to your house (they are simply folloowing a trail left by others) is to spray either some lemon juice or peppermint oil (be sure it it is real and not a synthetic) in that area and after they have moved on fill in those cracks to keep others from gaining entry.

Winged ants emerge in the spring and I suspect that that is what melonmagician is observing.

Can you calk around your window so that the ants can no longer gain access to the inside?

yes in fact they all have wings..already got bit by one...theres also a few male worker fire ants...but i have had to smash 20 queens so far..

I am on the 2nd floor even!!! I cant caulk around the window so i need another option