possibly blossom end rot maybe?
could you post pictures?

Too much water or lack of calcium.

I would say pick up some bonemeal for them. Peppers need alot of Calcium (especially for habanero or bhut varieties) for their leaves and peppers or you get blossom end rot. Since you said you haven't had much water on them I assume this is your problem barring something damaging them like a pest.

If your looking to save some money save some eggshells and bake them in the oven for a little while and grind them up.

Take either the bonemeal or ground eggshells and sprinkle liberally around each plant and water in.

Best of luck.

p.s. I have attached a link which has pictures of a pepper plant's leaves with calcium shortage...just scroll down and see if your leaves look similar.

Here is a link that might be useful: Growing tips