Admittedly, it's hard to kill another person by voting, or receiving due process. And we do restrict speech when it verges on violence (imminent lawless action, fighting words).