Digger, that is indeed a lot of clay. Other concern? Your proximity to the tree and it's roots (in picture behind your hole). I can see you've cut roots to dig your hole.
Asparagus is a fan of afternoon shade, but it is not a fan of too much clay or things like sharing nutrients with tree roots. Asparagus is a piggy.
Frankly, to avoid all of that I would simply do what a raised bed is meant to do and stick it on top of the soil. No digging. Bed will drain around the sides, frequent amendments of compost will eventually work their way down into the soil over the years creating even better drainage.
I put my asparagus in raised beds directly over the grade (it is all clay under there. Heavy layer of yummy compost sharp sand, top soil and manure, on goes asparagus, then I topped it up and have since topped it again with some compost and mulched it all in heavily. Asparagus seems happy even in 107 degrees of baking sun.
Make it easier on your self :)
In the future to start on your new beds, start early! I'm beginning to do that. Choose a place and begin laying on organic "stuff" layer by layer. By the time you want to plant in a year or so, you can dig in there like butta!!!

the roots are from a sweet gum that was between the bed and that pine. Its gone now. Don't tell me all that diggin was for naught, lol. How would be the best way to fill the hole. I've got plenty of manure. Should I use any of the existing clay? Add gravel, sand or what? I am gonna build the raised bed up at least a foot and plant in the early spring. thanks

mix the manure and clay, then throw the wooden bed on top and use topsoil, manure and a rough sand if you can find it. Don't use play sand, it's too fine when it comes in contact with clay it can make compaction worse. If you'd like to add compost to all of that you can do that too. There are explicit directions for planting asparagus online, it needs to be planted a certain way.

Either fill the hole with some water or watch what happens after a hard rain. If it drains within a couple hours you are fine using the hole as is.
Asparagus likes a lot of organic matter and add some 10-10-10 in the spring.

with all this rain that has been around lately, Coweta county, or should I say "my particular house" has gotten very little. Yup, I think I am going to haul some of the clay back and mix it with my manure. Once I fill it back to grade I will build a bed and put it on top. I'll use manure, topsoil and some river sand for the bed. Then plant the asparagus in January. I was thinking of a 16" bed? Would that be overkill or necessary?

Digger, 16" is generous, and it would easily work. I think at least 10" for asparagus in an area that has less than optimal drainage is the minimum, simply because it likes a deep planting.