Peat moss works fine from all I've heard, especially if you mix it with finely-shredded paper to increase the bulk and reduce the cost.

I'd say the pros are that it's readily available (you can get it at any garden center), it's not very expensive, and it absorbs liquid well. The main cons are that you do have to pay for it (shredded paper is practically free), and it's less environmentally friendly than some alternatives. This last is because peat is not a renewable resource. It's what left after a bog or swampland dries up and the sludge from the bottom sits around for a few thousand years. So it's actually "mined" out of the remains of ancient wetlands, and we can't produce more on any time scale that any of us will live long enough to care about.

My advice would be to use as little of it as you can, by stretching it out with shredded waste paper and shredded cardboard. Consider coconut fiber (also called coir) as an alternative. It has all of the advantages of peat but it IS renewable. It's actually a waste product of coconut farming - you get it every time you grow a coconut. Of course you have to pay for coir as well (it's about the same price as peat), so if you're cheap like I am then you'll want to stretch that out with shredded paper as well.

Actually what I do myself is run both a hot composting bin and a worm bin. I take the compost that comes out of my hot bin and use that as bedding for my worm bin. The worms love it, because it's loaded with the bacteria and fungi that they like to eat. You'll want to mix some shredded paper into it as well, I've found, if you plan to add food scraps to it. But what comes out is pure black gold.