I dont think the delta-v for deorbiting is even necessary. You just need to agglomerate all the debris into one big mass. This will make it easier to track.

In the future, when we need some in-orbit resources, etc, we can use these. (much of the debris is spent booster stages)