It used to confuse me too..now I think I get it.
The issue isn't all thermometers going up by 3.5 degC. The issue is that it also brings a host of other changes, and it's those things that kill you. They include:
- higher variability of temperatures. So your +3.5 degC actually makes heatwaves of +10/+15 more likely
- changing rainfall patterns, taking rain away from some places that rely on it.
- rising sea level, where the small average temperature changes are sufficient for a significant increase.
I actually think the focus on global average temperature increase is a major failure of public science communication, precisely because everyone mentally adds the increase to their local climate and thinks "meh".