See if the info at the site below will help. Also remember there is a valve in the priming bulb sys that works as a one-way valve (to force fuel one-way that you might need to check.

Looking back at your post, I feel the diaphragm you replaced is not related to priming vs fuel delivery while running vs starting. Have you looked at a diagram to know the diaphragm and gasket are assembled correct? Double check your diagram or there should be a youtube or diagram on line to show the assembly.

Posting a Pic of your carb question should help.

http://www.youtube.com/watch?v=uCC1qO75zDk

Here is a link that might be useful: Primer Help