I'd like to grow my own nuts, but I'm on the edge between zones 4 and 5 and I'm seeing conflicting reports about what nut trees will thrive in this zone. I'd welcome any advice/reports of your experiences.

I've had some success with shrubby hybrid hazelnuts (lots of nuts, but very small). I'm not fond of black walnuts, though I'd be glad to have other types of walnuts if they'd grow here. I've read that butternuts are winter hardy to zone 3 but prone to losing flowers in late frosts and also susceptible to fungal and viral diseases--do you know if butternut disease is common/present in this part of the country?