Yep, that's why it's so scary. By the time the effects have become so obvious that even the strongest deniers realise something must be done... that's too late.
And it's not just about temperature. It's about the socio-political effects of climate change. How many people realise that a large part of the Syrian conflict is actually down to crop failures? Migration from country -> town led to big social conflicts, which massively contributed to the eventual military conflict.