Oakleafs bloom primarily on old wood on terminal stem ends, and set buds for next year in late summer/fall. Either your winters are too hard on it or deer are eating the dormant buds.

You might get a few blooms late if the terminal buds are gone, because the topmost lateral buds might decide to flower. This is very intermittent, however.