Surviving death records from Ancient Rome are far from representative. Contemporary record keepers were mostly interested in heads of households, and not, say, slaves. So this again skews the statistics towards richer and more influential people.
Also, Rome itself (as a city) had never problem supplying its population with food. There was a special office (Cura Annonae) that coordinated import of food from Hispania, Africa and Egypt to Rome so that the city was never threatened with starvation. I just recently visited Ostia Antica, the main port of Ancient Rome; it was absolutely massive. Rome certainly had a lot of problems, but it was fairly secure in food.
Things were different in 11th century Germany or Sweden. Without an extensive network of long-distance trade, if crop failed in your region, there wasn't any way to prevent famine.
Even deep in the 18th century, during the last peacetime famine in Czechia (1770-3 approx., caused by rainy and cold weather), the attempts to alleviate the hunger by importing food from Hungary were only partly successful; even though by that time, the road network was fairly developed and the state capacity was orders of magnitude higher than several centuries ago. As a result, something between 5 and 10 per cent of the population died.