1. Pedestrians move at much slower speeds than bicycles. If two pedestrians walk into each at 5mph the damage is much lower than if two cyclists collide at 30mph.
2. Since pedestrians move much slower than bicycles they can manoeuvre much faster. The stopping distance for a walking pedestrian is less than 1m, the stopping distance for a bycicle travelling at 30mph is much greater.
3. If two pedestrians walk into each other the chance of collateral damage is quite low. It's quite easy to walk around two people that just walked into each other.
If two cyclists collide, there is a good chance that cyclists on each side of them will get caught up in the collision, crash and injure themselves too. A good example of this are the crashes seen on Tour de France.
Of course, if people started running as a mode of transport it would become increasingly dangerous, and you'd have to start separating the walkers from the runners as the damage from a potential collision would be much greater.