Those are called seed leaves, and it is common for them to discolor and drop off as the true leaves appear. It happens w. many different veggies

It is not normal for the seed leaves to turn yellow that quickly though. I grow dozens of peppers every year and the seed leaves usually remain green for a long time. You may need to feed the plants. A liquid fertilizer that has a fair bit of nitrogen would work. Anything like Miracle Grow or Fish Fertilizer, etc. would be ok.

Another possibility is that the seedling have dried out way too much at some point. When that happens sometimes plants shed the older leaves, which usually turn yellow in the process. This is uncommon with seedlings, but possible.