Fill them with a beautiful colored glass or small stone tiles

Well, clearly you need something higher. You could either hang art on the walls of each (which would allow a couple of little knick-knacks to remain) or go for something tall on the ledge, like a vase with willow, or a spray of artificial flowers, or a large plate , or somesuch. (Would need real dimensions to talk in more than generalities.) Depending on the drama of what you choose, you could paint the backdrop a slightly darker shade....

Looks like it would be a good space for a frame collage of photos. I've seen them done down stairways too. It wouldn't have to be formal either. Different size frames all around the space. Make sense?