In my opinion, depends how close you are to planting outside. I am about a month from planting outdoors here, and I have been pruning back plants that have set flower buds (so far, 2 plants have been pruned, out of 10 plants total). I might continue to prune back plants that set buds for another 2 weeks or so, but once I'm within 2 weeks of planting outside I think I'll leave them be.

Before I planted mine out this year two of them already had nice size peppers on them. :)

I have flowers on probably half my plants, and peppers on at least three. I still have 3-4 weeks until planting out. I considered snipping them, but figured that if maintaining flowers and fruits retards growth a little, that's OK, as I will run out of room if they grow very much bigger now :)

Thanks everyone!
Since I am going to be growing them in outdoor pots I think I'm going to go ahead and let them flower and start hardening them off now...If we get a late season frost I can always bring the pots in.
Thanks again!