This is a correct, but incomplete take that I see a lot in response to the "market is going to crash, yay!" comments. It's incomplete because it assumes that every housing unit in the country is owned by a single family and, if sold, would have to be replaced by another one. These are owner occupied primary residences which are 70-80% of the market, depending on who you ask.
The remainder consists of second homes and investment properties which will get sold, if they don't make financial sense anymore, for example, when short or long term rental income drops which is already happening. Overall, it's possible for the prices to drop in the total market even if the primary residence owners swear that they will take their 3% mortgage to their grave.