Quite a few of the pepper plants I planted in the ground in the last few days have put on many (I mean many) buds. Some of them had buds even in the starter pots. I am not sure how to tell if it's stress or normal. I am showing some pictures below of a couple of these plants with a pair of clippers for scale. Are these plants still too small to set bud? Typically how big would you let a pepper plant get before you let it flower?

Sorry, pict quality not so good.

Lipstick Pepper, Buds

Sweet Thing Pepper, Buds

Should I do my best to remove as many of these buds as possible or can these pepper plants start flowering and fruiting now?

Thanks!