I just posted the link below in another thread, but I guess it won't hurt to post it here, too!

The hole in the fruit sounds like insect damage to me, not a disease. Maybe thrips or armyworms, but those are just wild guesses. Keep an eye out for insects and take pictures if you can. At least notice them carefully so you can remember and maybe find them online somewhere.

Here is a link that might be useful: Know and Manage Pepper Pests (and Diseases)

Check out pepper weevil linked below. The photos will help you ID if this is your problem. The adult drills the hole and deposits an egg inside the pepper. The hatched larvae then thrives on the inside of the pepper and leaves the mess you found. Info on controls is also offered on the link. Good luck.

Here is a link that might be useful: Pepper Weevils