Telling someone to shut up because they're being abusive is still speaking up and taking responsibility. You have all the right to say what you want. Nobody has the right to listen to you, and they have as much right to tell you to shut your hole.