Since Damore explicitly said that he thought that the women working at Google had passed the same bar as the men, and hence are all qualified to be there, anyone who say "Damore wanted women to get paid less" or "Damore wanted women to get fired from Google" or anything like that just didn't read the words. They might have read the memo, but not the words, they saw that some of the things he said resembled alt-right talking points and then assumed that he had all the normal alt-right beliefs.
Note that I still think there are some questionable things in the memo, but the way people demonize it shows that they can't read controversial texts or didn't read it and just relied on second hand information from those who can't read controversial texts.