You can dig out the rock and wash it off.....but that's a lot of work.
Do you by some chance have plastic or landscape fabric under those rocks? That would keep the clay bits from sinking below the rock. If you have something under the rock, I suggest you remove it!
Sometimes you try something and don't get the results you hoped for....this is one of those times. I am guessing that you put the river rock there to provide a dry creek so that after a heavy rain there wouldn't be erosion of the loil. Well you accomplished that, but the rocks are getting covered with red Georgia clay.
If the answer isn't remove the plastic under the rock....why not plant something in the rock thatw ill disguise the clay....something like a sedum.
Linda C