Terrible stuff! What happens is that very fine particles build up on the surface over time, impeding water infiltration, gas exchange, and evaporation. I'm so sorry that you are having to deal with the problem....there's nothing to do but remove it.

My recommendation (for many years) is that homeowners rip it out and that professionals stop using it.

It does have useful purposes in permanent garden walkways, where soil health doesn't matter.