Your rubble foundation can be re-pointed.

To prevent the accumulation of water is another matter--typically interior or exterior drainage tiles. See here. (This repair by the foremost building scientist in North America on his own house also includes replacing the rotted sill plate.)

Many fieldstone foundations are held together with mortar. Some older foundations are actually made with loose fieldstone (which usually get mortared over later).

The "sand" is likely degrading mortar that is disintegrating back into the sand it was made from. Mortar would help ... get a mason to come look at it and repoint it. You could also look into some of those basement waterproofing companies, just make sure they do stacked stone and not just concrete.