Not poisonous - the peppers are used in various ways and consumed, as is bark and leaves. You can certainly compost.......but since this is a borderline invasive species, I'd want to be quite certain things were properly decomposed and "cooked" before applying it to the garden.