Humans have still suffered far worse. At one point, around 70kya, there were only an estimated maximum of 10,000 humans, and it is from that population that sprung forth 8 billion today. I highly doubt with even all the damage climate change will cause that we will go back to such numbers.