That deep underground dirt is dirt..not soil.
Yes, if you want to give your yard half a chance of growing and sustaining grass plants and trees and other landscape plants you need additions of topsoil.
How much depends on the results of a soil test you get done on the dirt you have. Don't scrimp.

I'm concerned for you.
Let me tell you a little story about stumpholes.
Back in 85 I had a house built on land that was mostly clear so the few trees needing removal were pulled and carted away.
I watched the lot next door be cleared, the stumphole dug and the tree stumps dumped and soil packed and graded over it.
It was a spec home ultimately sold to foreigners who immediately put it up for rent.
Back then, stumpholes didn't have to be disclosed by the builder or the subsequent property seller.
Me, the nosy neighbor, disclosed that stumphole to each subsequent renter and ultimately the buyer of the property.

Long story short, my neighbor's child fell through the soil(that had been sinking for years as the stumps rotted) and thank goodness neighbor was in the yard when the child 'disappeared'.

Mark your survey with what you remember as the outline of those stumpholes and don't try to plant anything of height or value over them.