I wish that was the case... we could have so many more plants to grow.... Actually the hardiness zone map is based on yearly record low temps in a certain area, not average low temps... Where I live the coldest average low (since records were kept) is about 18F in the middle of January. The coldest record low temp in my area is about -22F in Jan of 1994. During the winter of 2001-2002 it failed to get below 8F.
When the 1990 Version of the hardiness zone map was completed it compiled record low temps for each year during the 1970's and 1980's (A somewhat colder than "average" stretch). For instance if you had record low temps of: (Just an example)
1980 -2F
1981 -12F
1982 3F
1983 -10F
1984 -1F
1985 -20F
1986 -5F
1987 4F Average: -6F (zone 6a)

It really is not a perfect system, because often if you live in zone 6, and you plant according to zone 6, you often have winters that could be zone 4/5 winters (1985) that would damage your zone 6 plants, but also you would have winters where the temp fails to fall below 0F, and you could have zone 7 plants that could sail through unharmed.
Also, if you live in the city where recorded temps are usually NOT taken, you could be some 5-10 degrees warmer on average than the surrounding countryside (due to the heat island effect) The key is to know your property, its microclimates, windpatterns, and winter sun angles when planting. I have had supposedly zone 5 hardy plants killed to the ground when planted in a low spot in my yard- where cold often collects... On the other hand I have had zone 6/7 hollies, camellias, musa basjoo, hardy needle palm, etc. survive the same temps because they are planted near the house, close to brick, on an elevated part of the yard, or facing the SE (where heat is retained).
There is a tremendous amount of trial and error in gardening, and eventually you learn what works, and where... But, it is always fun to try "marginal" plants and see them survive where the are not supposed to.