No, to say that climate change
is a factor, you just need a non-zero correlation between climate change and any of these events (i.e. even if it only causes a 0.1% increase in weather variability and that contributes to an increase of only 5M galons of water in a flood, it is still a factor). And peer reviewed studies are plenty and they do support the idea that a strong correlation does exist across many types of extreme weather events.
If you want to argue that carbon offsets don't have as much direct impact on specific problems (floods, fires, whatever), sure, that's a valid question about resource prioritization, but the theme I keep hearing is that these things keep getting worse year after year and that in a lot of cases it's no longer at a scale small enough for infrastructure projects to even be feasible anymore.