Most often the insect pests that create problems stay on the plants so if they are gone you may have predators, beneficials, that are helping you control those pests.
There probably is nothing you need do, at this time.
There are a large number of reasons a plants leaves turn yellow, chlorosis from nutrient problems, the leaf stem lost its connection to the plant and is not a working part of the plant, the leaf is too shaded to manufacture nutrients to supply the plant.

Anybody got an idea as to what could be causing this and what I can do to get rid of these pests so I don't loose my 2 other pepper plants.

ID would be hastened with a picture of what is eating plants. Yellowing likely not related to insects but rather some environmental factor like kimmsr mentions.

Dan