If you read his latter books, Diamond makes it quite clear that he really does believe that the environmental situation of a culture is the most important factor in its successes and failures. (Collapse is especially egregious in this regard).
One of the main problems of Guns, Germs, and Steel is that it ignores a lot of evidence that is challenging to its thesis. To be fair, some of that evidence wasn't available when the book was first written, but Diamond's more recent comments indicate that he still believes in now-discredited theories (such as Clovis-first peopling of the Americas), and some pieces of evidence are fairly obviously wrong even when he wrote it (such as the fact that most latitudes don't actually have climactic continuity, and the ones that do involve civilizations that were "backwards" for most of history, such as the steppe nomads).