Have you been giving it regular drinks? A mature rosemary can tolerate some drought, but a young one needs a bit more TLC (and water). Drainage must be very good, of course.

How often do you water it? How deep?

I live just north of you in San Marcos, and I've also killed a few rosemary plants by not watering them enough. I finally have one doing well because it's gotten plenty of water while it's getting established.

It took me a while to figure out what I was doing wrong because there are huge rosemary plants around here used in landscaping, so you'd figure they'd do fine. But while the plants are young they can get fried in the hot summer.

Hope that helps.