When it starts interfering with agriculture by killing crops, resulting in famine and food shortages.
As these sorts of heat waves become more frequent, our food supply will become more unstable which, left unchecked, will eventually result in famine even in rich parts of the world like Europe.