My theory is that really hungry deer will eat almost anything, and with a nibble here and a nibble there, really toxic plants are not ingested enough to deter them.
Starving deer will even eat red cedar, and the toxins in the cedar kill the normal bacterial populations all such animals need. The deer die, unable to digest what they have stuffed into their stomach, and of starvation.
Interestingly enough, they do not seem to eat grasses.

I've never or seldom seen deer damage on Illicium, Clerodendron, flowering quince, Weigelia, Spiraea, Lagerstroemia, Cephalotaxus, Tsuga, Pinus, Poncirus, Begonia grandis, Zantedeschia, Crinum, Opuntia, Cornus, Bignonia, Gelsemium, Buxus, Ilex, etc.

They avoid aromatics--some of which are known for beautiful flowers as well (lavender, and rosemary where hardy), you can always plant the hardy cactus which are covered in beautiful yellow lowers in June. Other flowers that are same are bleeding heart, columbines, day lilies, daffodils, iris, lungwort and rhododendros. There are lots actually. But sometimes I will find things damaged that they are not supposed to touch (but may not always just be deer!) Flowers to protect from deer--azaleas, hibiscus, camellia, roses, hydrangea, HOSTA. Mine seem to love Euonymus japonica best of all.--I give p on that one. Would move them into the house but the deer would only be breaking into the house to eat them!

Thanks, everybody. Some good ideas here. I need to try the hardy begonias again, for one thing. Unfortunately I don't have enough sun for most of the aromatics (or the cactus, probably) but I should have added to my list Caryopteris. My deer have never touched that.

WW

Try Queen Cleomes (Cleome hassleriana)! The deer stay clear of them, probably because they have needle-like spikes on the stalks. Though they are annuals, mine have reseeded for years. The spidery flowers in whites, pink and purples last through the summer. They can grow quite tall, depending on the variety. Mine grow to about three feet tall, just take that into account before you plant!
Good luck!