You're right, speech should not be limited... in fact I am telling everybody about the time you beat your wife and abused your kids. And I'm putting $100,000 in to advertising this all over the place and ensuring every forum is littered with this fact along with your name and address...
Hopefully you see simple solutions come with their own complex problems.