As you note, there are a ton of confounding factors. Growing populations and increasing wealth mean that the same number of storms would be expected to result in more losses now than before.

This seems to be the crux of the dispute between the author here and his critics. The author has analyzed the data and come to the conclusion that the increase is entirely due to these non-weather changes. His critics see that some of the increase is left over, and thus due to climate change, after accounting for all other confounding factors.

Note that a small change in the climate could result in a large increase in the number of loss events. For an example I find easy to understand, consider the influence of sea level rise on tropical storms and hurricanes. Making up some numbers, imagine that an area suffers major damage once the storm surge hits 10ft, which is expected to happen in 1% of storms. If the sea level rises by 1ft, then major damage happens at 9ft of storm surge. Since these events can be expected to follow something like a bell curve, we can expect 9ft storms to be far more common, maybe 10% of storms.

Again, numbers totally made up, but the point is the idea that small changes in severity can potentially result in a large change in frequency. And this combines with other effects, like climate change potentially pushing the distributions farther out (so that perhaps 10ft storms become vastly more common, because what used to be a 9ft storm gets slightly intensified) or a shift in severity causing a net increase (worse storms in region X plus milder storms in region Y could result in more total storms being over some damage threshold).