The only concern with fresh cow manure is that it might heat up so if you have it and the worms in a confined space, you could cook them - age the manure first to ensure against heating. Also, if your worms are in a bin (vs. outdoor heap or windrow) that shares your living space, it might not smell so good!
Most barnyard manures are okay, however chicken manure can be very 'hot' and should be aged for quite some time or otherwise composted.
Finally, horse manure from horses routinely treated for worms should be well aged.

Thanks. That was helpful. Couple of more questions :-)

1. How long (in days) should one age the dung (from each animal) before putting in the worms. Is there a way to come to a conclusion by visually inspecting the ageing dung.

2. Is it a good idea to mix dungs (homogenous mixture) from all the animals and allow it to age together as against ageing dung from each animal type separately.

3. Would it be ok, if I put the dung in the pits and allow it to age ?

4. I believe that with time, the dung would loose moisture content and become dry during ageing. Should it be watered or left as it is ?

how long it takes any manure to loose enough N to be sure it won't start heating depends on the type and if it has any/much bedding mixed with it .... "ageing" is actually pre-composting, to allow the material to get past the initial stages of decomposition, you should be able to recognize when the process is well under way :)

pure fresh cow, pig, or chicken manures are higher N, and really should have some straw/sawdust mixed in - pure horse biscuits, and most small animals like goats, sheep, rabbits, are less of a problem, and in small amounts can be used fresh - if you have all types, mix it or not, makes little difference ... and yes, don't let it dry too much

Bill