I don't think this is an unfair reading. One of the bullet points in the conclusion is literally that the company's diversity programs are problematic because they alienate non-progressives.
But actually doing so is a fantasy. Building bias-free selection processes is really, really hard. I don't think anyone actually knows how. (At least not useful ones. You can trivially eliminate bias by never hiring anyone, for example.)
It looks to me like this is the fundamental agreement between the two sides here. Both sides want to hire the people who will be best at the job, full stop. People who think James Damore is correct believe that the typical hiring process, or at least some achievable tech hiring process, already achieves this, and any modification is necessarily selecting worse candidates. People on the other side, such as myself, believe that the hiring process selects for far more than just merit, and diversity programs are an attempt to fix that.
So, yes, let's pick the best of the best. How?
Diversity through quotas or hiring preferences to X, Y, Z classes of people are bound to make people angry.
Diversity though increased access to education, recruitment efforts, and accommodation of different strengths and weaknesses, personalities, work styles, management styles, etc. would make people much happier.
Oh that alienates you? So you want to murder people and steal stuff? We are going to fire/blacklist you now as a thief.
"Politicization" is quite vague and the author doesn't explain exactly how these programs are "politicized" or what that means. I suspect it comes down to their mere existence.