Right - but what you just said was an expression of your risk/reward preference. :) But also, in the US, there's a pretty large contingent of mortgage holders who have <= 3.75% mortgages [1], which compare very nicely to the (expected / hoped for) 9.7% average return from a diversified total market fund. That's not 1-2%, that's an expected ~5%, depending on how things sit from a tax perspective.
(You don't need to account for inflation in that return calculation, since the mortgage rate is also affected by inflation.)
If you're me -- 42, great job security, relatively small mortgage relative to income, and in a high tax bracket that's unlikely to change soon -- it's a no-brainer: Take the risk and go for higher long-term expected yield. A recession just means I keep doing what I planned to do anyway - working and saving more money for retirement.
If you're 60, planning on retiring in 5 years (and so about to drop into a lower top marginal tax rate), and in an industry with uncertain job prospects ... suddenly paying off the mortgage looks more attractive from a risk minimization perspective. Or at least splitting the difference.
[1] Most people who took out or refinanced mortgages in Jun 2012 - Jun 2013, and 2016
https://fred.stlouisfed.org/graph/?g=NUh
The "or refinanced" is important, because when rates were that low, a lot of people had a strong incentive to refinance, so the actual distribution of outstanding mortgages is biased towards the lower rates. [2]
[2] This is a study from 99, but the point remains: https://www.newyorkfed.org/medialibrary/media/research/curre...