Hardly likely. What makes you think the underlayment would be toxic?

That’s actually a good question @kudzu9. I’m pretty sure it has asphalt in it, but I also just kind of assume most construction-related materials have some level of toxicity / carcinogenic propensity.

Asphalt and water don't mix. Any asphalt that might leach of the roofing material will be filtered by the filtration system. Besides, the chlorine in the pool water is far more carcinogenic than anything leaching off the roofing debris that fell into the pool.

Assume you are talking about the asphalt impregnated felt paper, not newer synthetic products. And that is it just a few cut "shards" like smaller than your fist and were removed within a couple days. If so, and I am not a chemist, but I would guess the dilution rate for your size of pool (unless it is a kiddie pool) would render the "oils" benign. At the same rate if someone peed in the pool.