It may be because you do not get enough Winter chill. Pruning them back in late December may help them go into dormancy and bloom better later. (just prune the bud tips and remove leaves)

Thanks for the reply, lack of chill is possible but of all the suburban areas of the LA area my valley probably gets more chill than any other area. And it's not just the blooming, the plant is not growing. No new growth at all since I planted it more than 3 years ago. I was at the LA county arboretum in Arcadia several weeks ago and saw several of them thriving there.

Chill isn't the problem. I grow it and I get a lot less chill than Crescenta Valley. It needs quite a bit of water though. Mine got hammered this year because it was in the 70s one week and it would push out new leaves and then it would be 100 the next week and all those tender shoots shriveled up.

I planted one many years ago in Sacramento. My experience was that it is very slow growing. It seemed to not grow much for a few years before it finally took off. Love the scent!

Mines over 10 ' tall here by Disneyland no chill. They are super slow. Baby them along on the best soil you can give don't add too much amendments when you plant which can rot and cause root problems. They love regular water but need to not be wet all the time. Mine gets morning sun and late sun but is shielded from 1-3 by a tall Italian cypress. They smell divine. Give good quality mulch on the top or a good layer of leaves to break down and feed them. I feed some inexpensive lawn food twice a year. It also eats banana peels, melon rinds, carrot tops, old lettuce, whatever I have to give. I pull the leaves back and put the scraps down and cover it up again. The worms do the rest.