Why not drywall over the concrete?

The item you want, panels of plastic with subway tile design commercially exists.

Install furring strips, some may need to be scribed, attach panels.

I would install dense foam under then panels to provide some insulation and ridigity.

Shiplap could be pretty in a small bathroom. The wide planks look like a design decision and cover the offending concrete.

A Mat

Do you have a name of a product that I could look up?

PVC glue up wall panels

You need to frame it out and insulate it first. NEVER EVER apply anything straight to the basement walls.

Finishing a basement by furring out walls opens a can of worms - things like condensation, insulation, water infiltration, butting up to the shower surround, etc.

Avoiding all of that for just a more finished look, some 1x furring strips with hardboard paneling attached might be a sufficient half-thought solution. You can find decorative 4x8 hardboard panels at your orange/blue/green big box store in stock, a couple with a white tile pattern look (they have an acrylic/vinyl cover on the hardboard).