When wielded properly pencils are indeed dangerous, even more dangerous than swords in some cases. But less dangerous - in the general case - than say a gun or a knife which is why we call those weapons in the first place.
Note that according to the definition of a weapon ("a thing designed or used for inflicting bodily harm or physical damage") a pencil can be classed as a weapon due to its use. Which is a perfect illustration of why men with pencils (used as weapons) are more dangerous than men without.
So thank you for making my point.