The same thing is happening to my Sweet Shell Peas. I sowed them directly in the ground about 9/1. My guess is the weather was way too hot. I did use a shade cloth but the weather in the first two weeks of Sept (if in full sun) was extremely hot. I could cool it down to under 90 degrees F under the shade cloth if I misted a couple times during the afternoon. I guess it was too much stress on the poor little plants. IF the sun would come out (instead of overcast) I might get some peas IF the weather does not get too cold too fast. It is not recommended planting peas in the fall in my area but I am always trying anyway.

I would be hesitant to pick off the flowers at this time since I'm guessing you don't have very long until frost. It is possible that they flowered early due to the heat (though mine are now almost three feet without flowers and we have had 2 weeks of 100 degree heat since I planted them. The worst I have had is a few leaves dying). Personally I would probably just let the flowers that you currently have go to fruit and hope to get some more.