Cinder Blocks will be your best choice if it is going to support a lot of weight. HD offers bulk discounts for a lot of building products, I find them to be as cheap as anywhere else.

CMUs (what you're calling cinderblocks) need to be built on a concrete footing, mortared together, tied to the footer with rebar, and the cells filled with concrete. Is this what you're looking to do? If so, I've found that the 8x8x16 CMUs from Lowes are ~$0.40/ea cheaper than my contractor yard. Occasionally you can find someone on craigslist clearing some out for a buck apiece, but it'll take you a while to get what you need.

Or are you referring to the interlocking concrete wall blocks, that go on a gravel footer and lock together with pins or a lip? If that's the case I wouldn't even consider the ones from the big box stores as they're trash. You want to buy those from a paver and stone supplier.

If you are trying to retain soil, which you are, please do not use cinder blocks. Use retaining wall blocks that have a setback to them - meaning each row of block you stack on top of the other one, it sits back about a half inch. If you stack anything up vertically it will tip over in the long run. You can find retaining wall block in a number of big box home improvement stores. Consult either a professional or watch some you tube videos on how to properly install a retaining wall. There's a lot more planning than setting blocks on the ground.