Need to know your geographical location before we can make any plant suggestions.

How large is the area in question? How much sun?

We're in Arizona. The area is about 200sqft and it gets full sun throughout most of the day.

I got this from Grok.

For a 200 sqft, full-sun Arizona backyard, Kurapia is the best walkable groundcover due to its drought tolerance, moderate foot-traffic durability, and cost-effectiveness. Pair it with buffalo grass for a soft, lawn-like area and add desert marigold, penstemon, and a small agave for color and structure. This combination minimizes water use, handles Arizona’s heat, and creates a functional, attractive space. If you prefer a more plant-heavy design, reduce grass and increase groundcover or gravel with native perennials. Drip irrigation for 1-2 years, then hand watering or rain. Check with Arizona Native Plant Society and Desrt Botanical Garden(plant sales).

I hope when you say fill in you plan to have a little breathing room as crowding plants cheek to jowl in the desert isn't a good look IMO. Some plants we’ve had success with besides cacti, agave, and yucca are desert

desert milkweed

Chuparosa

Parry's Penstemon

Desert Mallow in the wild

Flattop buckwheat

And I hope you saved some of that rock for mulch!

Some option