They don't like wet feet! See if you can improve the drainage or move it to a well drained location. I recently lost a 'Harmony' that was planted in sandy loam, on a slope.
When I dug it, there were no substantial root system, only a single root, about the size of my little finger and 7-8" long. Not nearly enough to support a 3 ft. shrub. For some reason it never developed below ground as it did above.
Rb

My oakleaf is having the same problem. I have it planted near a gutter drain and the ground is just too wet. It is four years old, but I think it will probably not make it through the summer. I plan to remove it and put a beautyberry or Virginia sweetspire (which tolerate moist/soggy soil) in its place. I will plant another oakleaf in a drier location.