The powdery mildew isn't helping things much :-) But I'd suspect insufficient water rather than too much. That could also account for the PM as well.

And water at the base. Don't get the leaves wet.

Actually, PM spores cannot survive in water. Hosing down or spraying the foliage will often prevent the development of PM but it won't do much for already infected leaves.

Seems like every time I use the hose sprayer on the leaves, my stuff gets PM.

Might have something to do with how bad the humidity was at the time. idk

Humidity and lack of air movement can be big factors with PM. The tree looks a bit tucked into other vegetation. If you can move it to a location that gets better circulation and not too much shade, that would help.