I prune my plants mid winter when no growth is happening.
If all the leaves fall off, any time after that and before new
growth starts in the spring.

And you'll find that after you cut it back, it will grow bigger
than ever. You can get 3 or more new sprouts on each branch.

Don't be timid either. You can cut back just about as much
as you'd like.