The problem is that if a collision happens it produces a lot of debris on unpredictable orbits. This process can run away and lead to orbital denial: https://en.wikipedia.org/wiki/Kessler_syndrome