When you make a case against a burglar, you don't base it on his views on capitalism and communism. You'd be laughed out of court. You base it on his act of burglary, with evidence/witnesses.
As far as the letter is concerned, it seems that its authors aren't really concerned with harassment. It's those other issues that stir real passion in them.
It is the point of the article. Author of the article didn't object to rejecting Stallman, and neither did I. They objected to doing it on the grounds of his beliefs and gave a wider perspective of the problem. If it was the first time something like that happened, I could say that it sets a dangerous precedent. Alas, we're long past precedent, we're just going deeper.
To be clear, there were other letters calling for Stallman's removal, such as the one from Debian, which don't have that flaw.
This article isn't about Stallman. It is about bigotry.