I used to 100% feel the same, but at some point I realized the problem was me, not him, in not viscerally understanding his goals. His stated goals are very clear, but the audience usually has somewhat overlapping, but nevertheless distinct goals. This is indeed at the very core of Open Source-Free Software feud. The base is almost entirely the same people, yet the ideologies are not the same, and in a very interesting way: the differences are
critical to RMS's ideology, but
minute to the other side. Thus, the other side thinks of a crazy guy ruining the whole thing for nothing or very little, and evaluates him as net negative for "the cause." Well, it is absolutely true, for their cause, not his.
I think his take on what compromises are valid and what aren't makes this clear:
https://www.gnu.org/philosophy/compromise.en.html
In fact, this particular incident, re Android, a seemingly "open" system, is a perfect example of the importance of his PoV in particular, as it illustrates that Open Source ideology would not have been enough to ensure the user is in control.