Can you post a pic, also seed are in the fruit so not sure what you mean went straight to seed? You may be seeing flower buds is my guess on the larger. Are you growing In ground or in pots?

I think Habajolokia is thinking the same thing I am. It sure sounds like we are growing something other that a pepper plant. A picture will probably be what is needed to make a determination.
Bruce

PS: Humanityneedspeace: Where did you get the starter plants/seeds?

how can a pepper plant "go to seed" with a pepper?

They can't. It is pretty clear he is growing something other than a pepper or possibly his use of the phrase "going to seed" isn't used in the way that it normally is.

I think a couple good photos would help clear this up.
Bruce

Sorry, this is my first attempt at a garden. here are some pics of the chile plants. They are in the ground not pots.

this is the full plant

The tall one does not look to be a pepper the small one to the right does look like one. We're they all sold as pepper plants or seeds?

Tall one is a weed. Yank it before it goes to seed.

the one on the right is a ghost chile plant, and the one was supposed to be a chile. I'll post a pic of the seed package that I planted the seed from. The ghost chile on the right is producing flowers, but the flowers are just falling off, and not producing either.. I'm so not a gardener. Thank you all for the help.

FYI I planted the seed in a seed pod. It grew in doors till it was about ten inches tall then i planted it in the ground. and that five foot thing is what it grew into.

It's possible that some foregin seed got mixed in with a manufacture seed packet. I think Jean001a is on the right track it does look like a smartweed variety you can Google image it.

Did you plant more than one Serrano? And they look different from the tall one. Also could you post a close up of the Ghost? The heat is probably causing the flower drop, how much sun does it get?

The seeds for the Serrano should have looked nearly identical to the Ghost Pepper seeds. Most pepper seeds look almost identical. But I know that plant is not a pepper plant of any sort. I would also pull it. Don't get down on yourself about it. We all make mistakes and likely that was not your mistake but the seed company's mistake. It looks like what you do grow is in great shape. So just start over this year or give it another go next season.
Bruce