Gingko leaves are used for tea, made from dried leaves collected in the late summer. (Sometimes it matters if you use fresh or dried.)

Some of the traditional medicinal uses attributed to gingko leaves suggest that it may have some blood thinning properties. Ginkgolides are being studied for their actions as an anti-histamine. (Another traditional use for the leaves.)

So I would approach consuming the leaves in any quantity with caution, but saving some for use as an occassional tea would probably be safe.