Probably end of winter will be best. One, you want maintain the advantage of the food-producing leaves it put on this summer, and two, you don't want to stimulate new growth too late in the season, although depending on your location that might not be too big of an issue.

I have a california pepper too and have found it to be so tough that you can do anything you want to it and it will keep coming back. The little shoots at the base of the trunk can be clipped when ever. But it is alway best to wait for cool weather before any major shaping or trimming