Actually a really good point. This is just a correlation with an imprecise causal link, even if global warming is clearly the most likely factor.
Direct observations of the warming over centuries are direct proof of climate change, in addition to proof in ice cores, tree rings, and sedimentary deposit patterns (among others) of prior rates of change being much more gradual.