All of that mostly depends on what you're going to be planting. Different plants need different pH levels which is what the lime is probably for.

I can think of two routes: either get a soil test and plant according to the test (finding which plant can thrive in the soil)
Or try to get a general idea about what you're goin to be planting, then amend the soil accordingly. Either way I'd say your compost addition is a safe bet.

The best soil "recipe" is to add about 6 to 8 percent organic matter to what you have. Epsom Salt and lime if a good reliable soil test shows a need for them but Epson Salt, Magnesium sulfate, will only help in very large amounts. Gypsum, maybe, depending on how sodic your soil is, something a good reliable soil test will tell you.
Adding compost, composted animal manures (horse, Quail, or any other) with 3 parts vegetative waste could be good for that soil, but only add "fertilizer" as necessary and shown to be needed by that good reliable soil test. Coffee grounds are part of the vegetative waste you can add, and the earthworms will come if you develop that soil into something they can live in.
What you want to have at the end is a soil with a pH in the 6.0 to 7.0 range with balanced major nutrients that can retain moisture but is well drained.

I'm going to be growing veggies. All kinds. I'm in southern CA so it will be a year round garden.

How do you get a soil test? Other than a couple of plants, this is my first time growing veggies in the ground. I'm used to containers and I know the soil is different.

AniaT: Yep.. lotta DG with clay here in SoCal. I don't know who recommended lime, but it will raise your ph and with our soils typically on the alkaline side, not a good recommendation. The only time i use lime is for my container mixes. Gypsum is used a lot here! Manure is fine, but it would be best if you can make some compost with it. If you don't have the room to compost, I have found buying it, by the yard/truckload quite reasonable. Also, make sure you MULCH. Imperative in this desert!

MOST veggies(from my research) like it a little on the acidic side, like kimmsr mentioned. Soil test -- call your county extension. Worms--- just keep feeding the soil and they will come.

For my organic ferts, I buy those in bulk also. Bloodmeal(quick N) Cottonseed meal(slow N), Bone(very slow P), and Kelp meal(slow K). Also mix in some fish ferts now and then. Eventually you can turn your crappy soil into something pretty good with copious amounts of compost though.

Good luck.

Kevin

Get a soil test before you do anything - there is no way to know what you might need until you understand what you have.

Lime in SoCal sounds quite unnecessary.....soils there tend towards alkaline conditions as does irrigation water so adding lime just compounds the issue. You should be shooting for a pH of right around 6.5.

Gypsum may be appropriate if you have sodic soils, which does go along with alkalinity. Also a reasonable source for calcium IF that is deficient.

Epsom salts completely unnecessary UNLESS soil is magnesium deficient.

Compost and aged manures are always helpful as is most other kinds of organic matter. They improve soil structure, provide decent but not excessive nutrient supplementation, assist in moisture retention and encourage a healthy soil biology. But I would add no other amendments until a soil test tells you specifically what you might need.

Your University of California Cooperative Extension Service does not do soil testing, but they can direct you to labs that do.

Here is a link that might be useful: UofC CES

Awesome, great info! I've ordered a soil test kit but at least I know what I can add as I put the dirt back in the bed. Sounds like compost/old manure is safe as is the gypsum and "meals" mentioned.

I will skip the lime and salts unless a soil test tells me I need them.

Thanks! I'm a bit clueless on amending soil so this gives me a great idea of what I need to do.

So it was time to put dirt back in the first half of my ploty before the soil test stuff got here. Ultimately what I did is layer. First I put a good amount of gypsum in and watered it in. Then horse manure, then layered native soil mixed with gypsum with compost, quail manure and horse manure amending more heavily as I went up. towards the top, i put some bone meal, blood meal and kelp meal as well. Wish me luck :)