I don't think the worm bin plan will work. Instead, I would suggest making a slurry and using that to water your plants in the garden for as long as it takes to use up the clay you have. In addition to that, continuing to add OM should help the loam of your soil.

Of course, what do I know, my soil has a ridiculously high clay content, and I've spent the last 6 years of my gardening life (new house) trying to get the soil to be LESS clay, and more loam.

Ultimately, I don't think adding sand to clay or clay to sand is the best way to go. I think in both cases, the best option is to continue to add OM. Although the minerals found in clay can be a HUGE benefit to your plants.

Of course, if you are doing a traditional SFG, your own soil is secondary to whatever you used to fill your boxes. Did you use the traditional "Mel's Mix" or a personal variation?

I'm adding clay in my garden to my worm bin with the purpose of breaking the large clay piece down to smaller pieces. However, I notice that unless I break the large clay down to smaller chunk, the worms won't get to it.

I put cardboard over the worst of the green stuff claiming to be my lawn. Then the six inch tall wooden boxes. Then a thin layer of lawn sticks, and woody weeds. Then grass and leaves mowed together 12 inches high in the fall. Then in the spring I planted seedlings in good soil in tiny pots and put them right into what was a 6 inch tall, 4 foot by 8 foot compost pile. A horrible oppertunity for plants :-) but I was more interested in keeping the boxes there a few seasons and then moving the boxes and having grass grow in those areas. I even wondered if plastering the bottom, corners and sides of the box with an inch of clay would retain water in the hot summer sun. But I did not have that much clay.