They will continue to come back whenever new seeds blow in.

And if any of the weeds are perennials, they will come back from the roots every year.

DG is an excellent growing media for all sorts of self-sown plants. The only answer to the problem is persistence -- remove when you see the, and/or use herbicide.

Over time decomposed granite accumulates dust from the air and wee bits and pieces of organic matter as well as plant seeds that will now germinate and grow in that medium. Periodic disturbance of the surface can slow that down and flaming the surface can aid in disrupting the growth of any "weeds" that do germinate, although that does use non renewable resouces and is expensive. Herbicides could be used, although they too are made of non renewable resources and are environmental hazards, causing pollution of our world.
vinegar, much stronger then the venegar you use (3 to 5 percent acetic acid) for cooking or salt can also work but they tend to be expensive, too. The most environmentally friendly method of "weed" control is to pull out those by hand, A bit more work but that is also good exercise.