If 10 people on the internet tell you they want to kill you, how do you tell if 1 of them is serious and is actually going to show up physically?
The answer is that you can't, and there isn't a firm line where all trolling is on one side and harassing/violent behaviour is something different.
Many of us can heuristically decide that trollish behaviour aimed at us won't extend to our person/home/workplace, and doesn't need to be taken seriously. That doesn't mean it's a rule for everyone or an excuse for that behaviour.