This quote[0] sums up things well for me:
> Look through any r/history thread mentioning Diamond and you will see dozens of people who find our critiques pedantic, and that, in a general sense, Diamond’s thesis makes sense. This is a very difficult attitude to address, because it’s rooted in a fundamental misunderstanding of how social sciences work. The attitude evaluates the ideas of popular authors from a utilitarian, practical approach: if the thesis is useful and helps makes sense of the world, it has value. We researchers take an inductive approach: if your methodology and facts are wrong, your thesis can't be right, no matter how much it "makes sense."
> For this reason, I’m kind of sick of talking about Diamond’s theoretical bents and ideologies. The standards by which scholars and the public evaluate them are so different that we have to address an entire epistemological orientation.
The "Diamond is a racist" argument is not one that historians care about. What they take issue with is:
* not questioning primary source bias (e.g. he takes Pizarro, Cortez, etc. at face value),
* exaggerating the role that disease alone played, as opposed to how disease amplified vicious cycles[2][3]
* made-up facts[0]:
> In a world where conquistadors bested Aztecs with with guns and Spanish friars set up missions in communities devastated by plague, Diamond’s arguments would matter. But this is a world where Tlaxcalans bested Aztecs, and Spanish friars set up many failed missions before gaining a foothold and witnessing entirely disrupted populations fall to disease afterwards.
[0]: https://www.reddit.com/r/AskHistorians/comments/6meq1k/comme...
[1]: https://www.reddit.com/r/AskHistorians/comments/2mkcc3/
[2]: https://www.reddit.com/r/AskHistorians/comments/8nr7xd/
[3]: https://np.reddit.com/r/badhistory/comments/2cfhon/
(Disclaimer: I haven't read GGS, nor am I familiar with the underlying theses. I just find myself fascinated by the controversy around the book.)