I don't have an answer for you, but I have a similar problem with my table grapes up here in Seattle. Maybe between the two of us we can figure out what might be going on?

My grape vines range between 4-6 years old, and I have Einset, Himrod, Jupiter, Glenora, and Suffolk Red. No powdery mildew usually, but I do get lots of mites when the leaves first come out. The only info I've found on the mites suggests that they do minimal damage, and any cure would be worse than the problem.

I read in a wine grape book that over-vigorous foliage could interfere with fruit set, so I'm wondering if that might be happening with my grapes. The wine grape book said that about 4' of shoot with leaves was all you needed to ripen your grapes, and you could cut the rest of the fruit-bearing shoot (and any side shoots) off. In contrast, some info from Washington State University about grape pruning in the home garden said that each shoot needs 14-16 well exposed leaves - which in the case of my vines will mean shoots way longer than 4 feet. I trust the WSU source more, of course, since it's specific to the region and to the home garden.

I did notice some white webby stuff (like the mites produce) at the end of a couple grape clusters. I pinched those areas off.

Do your vines have mites? (mine do)
How vigorous are your vines? (most of mine are monsters)
Do you fertilize? (I don't; the soil is fairly rich)
Do you water? (I do)

You wrote "reach the size of a BB 2/3 of the fruit in each cluster have fallen off,"

No pollinated, usually due to adverse environmental conditions at bloom. In my area, either rain or unseasonal cold.