Those babies will be just fine there for the summer. If having them that close really bothers you, you can dig them this fall after the stems and leaves have yellowed and replant the bulbs.

I have the same situation and am digging and dividing now. Lilies can be moved at almost anytime.

I am also digging & moving now. We have cool temps so I know they will move fine. If in doubt you could wait until fall without the tiny lilies suffering.