Per this document it appears that (frequent) Delta second stage explosions were one of the most significant contributors. By 1981 27% of all tracked debris (of orbital period under 225 minutes) were from Delta second stage explosions. (I'm guessing these are considered high velocity debris). It's quite interesting that NASA was surprised by this (asking McDonnell-Douglas to find out why in May of 1981) considering how long they had been launching the Delta rockets and tracking debris. From the document, NASA officials continually approved work to track orbital debris, but often didn't give the reports administrative attention.