Brick is generally softer and more prone to breaking. Also will be slightly smaller than pavers. Stone will be irregular and require more careful fitting. That, in my opinion, are the only drawbacks of the three.

Decide the look you desire and go for it. The base is the same for all and is important that it be proper for your climate.