Hens and chicks - Sempervivums - are able to grow in minimal soil. You can find colonies of them growing happily in the cracks of boulder retaining walls in my area with virtually no soil at all.

I’ve had similar boxes and sun conditions as yours and a few I was able to grow successfully over the years: moss rose/Portulaca grandiflora, rosemary, sweet potato vine, basil, oregano, annual lantana, sun types of coleus, draecana spikes (I’d mix in), marigolds, Mandevilla (I’d find the ones sold in 4-inch pots at the nursery), zinnias. I’m sure there are plenty more, these are ones I’ve had good experiences with. I’m in a hot southern state (z8a) so did have to water mine every day in the worst heat of summer, every 2-3 days in milder weather.

Thanks for the tips!

There's no doubt that I would remove them entirely.

Do you want to grow perennials or annuals
You would be surprised at what all could grow happily in 4 inches. Depending on the season, you could grow pansies, petunias, verbena, torenia, begonias, coreopsis, nasturtium, columbines....