You miss that the kinetic energy is proportional to the square of the speed.
https://www.quora.com/Which-one-makes-a-bullet-dangerous-its...
The speeds at which even the smallest debris hit other satellites are immense, compared to what we are used to in our surroundings. International Space Station moves at the speed of 17 thousand (!) miles per hour. Even the small pieces of something else moving at the different speed or direction will be extremely destructive, see again the explanation about how the bullets "work."
Then consider the number of the pieces there and the fact that their movements are "unstable" (as in not keeping the trajectories that are always at the same distance from the Earth).
Only once you understand all that read about:
https://en.wikipedia.org/wiki/Kessler_syndrome