May have less to do with fertility than it does with water and heat retention. Amended soils tend to hold water and maintain an even temperature better than a very sandy soil does and many seeds prefer that.

Don't confuse compost with "fertility", at least from a nutrient perspective. Compost offers rather little of that, though you can get a bolt of nitrogen from poorly composted manure. But yes, amended soils have a consistency that holds water better than sandy soil, and can have better insulation value. Not sure where you are, but when it gets very hot, seeds can get fried in hot soil.

"I know fertile soil helps grow vigorous plants, but does it also make such a dramatic difference in seed germination?"

No. One can germinate seeds in sand, on paper towels or coffee filters, or even rockwool. And many of us do. Nutrients mean little to embryo germination as the seed coat and cotyledon leaves provide all it needs to develop to the full cotyledon stage and even beyond until the true leaves develop.

Dave

Sand drains quickly so the seeds probably weren't consistently moist. If a seed dries out during the germination process, it's toast.

With fertile soils with lots of organic matter, there is better soil to seed contact and better moisture.

Dave provided the best answer here. Seeds contain all that is needed to start life provided proper moisture and temperature levels are there. The seeds you planted that did not germinate did not have the right moisture and temperature to do so.