I think this could look great. It will take some fussing to find the right pattern for combining the various materials. I think the style you are describing might be best accomplished by letting the bricks and stones meander among the concrete, but I'd have to see the materials and the space to say for sure.

The only thing I think I might find jarring is that I'm not sure the slate will go with the brick and natural rock. Funnily enough, the concrete itself might look better; I'd consider it. In particular, if you are going to have gaps that size you will have a lot of soil relative to stone surface, and your ground cover will grow very well. In no time, you won't see much of your stones anyway, and the work of putting on the slate will have been largely wasted.

Sand does not need to be tamped. Tamping is for subsoil or granite base. Are you sure you don't want to excavate to put granite base in for proper drainage? I know you don't care if it's uneven, but that clay underneath will cause drainage problems, it will make a nice clay bowl to hold all rainwater and cause major shifting problems, especially with small bricks.

The reason I was going to recover the concrete blocks with slate is two fold: the blocks are currently splotched with paint and I do have slate elsewhere on and around my house. I had considered staining the concrete to be a darker gray, but I really like the slate look. As far as the gravel base is concerned, I'm not opposed to putting it in, I just wasn't sure it was necessary.

Are 2-3 inch spaces between the pavers too much?

My concern with filling the spaces with dirt is that it tends to erode more quickly and therefore reveal more of the paver....anyone with postives/negatives in that area?