Rent/mortgage is just one part of the cost of living[1].
Even if you have a place to stay and sleep, you still need to clothe, bathe, and transport yourself and eat and drink to live. Cost of living is literally what it costs to live.
[1]: https://www.investopedia.com/terms/c/cost-of-living.asp
People can still afford cost of living, the American economy just chuggling along despite criticism is proof of that. The question is whether people are happy about spending the money they need to spend, the answer to which is a resounding NO as evidenced by the chief motivator behind Trump's victory.
Cost of living is still affordable (there is no "affordability crisis"), but it's too high for anyone to be happy with (there is an "unhappiness crisis").
Also, the people complained loudly and clearly that price of gas and eggs are their chief concerns. Housing is also expensive, but housing is usually a one-time lump and/or a fixed ongoing expense compared to food and gas which are ongoing small and variable expenses that quickly add up.
This is probably an artifact of how the media works.
People are concerned about prices but only some people are concerned about housing prices. The people who already own a house like high housing prices. Meanwhile everybody has to eat.
The media tries to maximize viewership so when they run the pricing story they're talking about high food prices (which everybody hates) instead of high housing prices (which only the people paying them hate but the people getting the money like). Which in turn causes people to be more concerned about food prices than housing prices because that's what the media is always talking about, even if the housing prices are what's taking the biggest chunk out of their wallet.