> The number of people in slavery would have still decreased

On what basis are you making this assumption? "The fall of Rome" was not an anti-slavery uprising. It was gradual disappearance of the central authority (and all its benefits such as roads, law and order etc).

About 10% of England's population entered in the Domesday Book (1086) were slaves. Compare that with the Roman Empire where slave population (including Rome and all provinces) are estimated at 10-15% of the total.