EDIT: Let me give a bit more of a nuanced position, because reading my post back, it looks like I'm saying that Stallman never did anything inappropriate before this.
Of course, I recognize that Stallman is fucking weird. I'm not saying that parrot-fucking, foot-fungus-eating St. IGNUcius never did anything inappropriate before; we'd be here for a long time if we were to make a list of all the times Stallman failed to read the mood.
However, I strongly believe that Stallman is not a creep. He holds some odd views and is pretty autistic, and so he occasionally makes inappropriate advances. Add to that that Stallman has been a very prolific person for decades and you won't relaly have trouble coming up with a list of anecdotes that, when framed a certain way, make him look bad.
I've read quite a few of these anecdotes by now. Some of them look fairly innocent to me. Some of them are clear social blunders. None of them look seriously harmful or in bad faith to me. Even taking all of them together, believing that all of them are remembered 100% correctly and played out as portrayed without any additional context, I don't think the appropirate reaction is to remove Stallman from his position(s).
What ticks me off so much about all this is the way the media deals with it. Stallman is portrayed as some kind of sexist, some kind of patriarch keeping women out of tech. It's ridiculous; Stallman is quite possibly the most inclusionary person I can think of. Stallman has had an immense, direct contribution to making tech more open for everyone.
I can understand the criticism of Stallman. I don't even think he necessarily makes for a good figurehead for Free Software; he has poor social skills, unhelpfully rigid and strong positions, and I feel like he often conflates free software issues with other social issues he cares about. He makes for a better philosopher than a political leader.
In this case, he's being thrown under the bus to score social points. People in tech are very itchy to do whatever they can to seem more inclusive, and condemning Epstein and his ilk is a noble goal (that also makes you look good). Stallman's statements that sparked this controversy were naive and he failed to read the mood, but they were not unacceptable things to believe or say. He did not deserve to have his words twisted, get kicked out of his own organisation based on the strawman, and then have people say that he was always bad, anyways.