a couple of large outdoor poly rugs to add color and interest.

Curious about how sufficient your base is and how long following the build the patio was installed. IF you reside in a frost zone a patio needs several inches of gravel to both drain and adjust to freezing. Construction backfill can take up to 5 years to completely settle.

Our patio has experienced minimal shift in 25 years and we’re located in northern IL. Build was 4 years following construction with average 8” gravel plus a layer of bedding sand.

https://youtube.com/watch?v=H5t9G-Gujqw&feature=shareb

A properly installed interlocking paver system should not "shift." I suspect it has to do with your base or butting up against the retaining wall that is moving. As an alternative, a concrete slab that is installed on a subpar base installation or against a moving retaining wall will crack and heave as well. And polymeric sand is easy to fix - don't use it - use normal construction sand for joints.

Construction sand will prevent ants from making mounds in between the pavers? I thought the poly hardens so this doesn't happen s but by it feels I keep battling it. The bar is gravel and sand