I should probably have said the change in life expectancy is a weighted average, weighted by how far you are from the average. If average life expectancy is 80, removing a data point where somebody died at 40 has 8x the effect of removing a data point where somebody died at 75.