So the roads were not paved after the 4th century due to plague and not a failed civilized administration? Would you consider the Merovingians as civilized as the Phoenicians?
You can't raise taxes when half or more of the population is gone. As per WP: "Some historians believe the first plague pandemic was one of the deadliest pandemics in history, resulting in the deaths of an estimated 15–100 million people during two centuries of recurrence, a death toll equivalent to 25–60% of Europe's population at the time of the first outbreak"
And this is after a couple generations of political turmoil, other plagues, and invasions.