If we choose to make having a home a priority, then we need to choose affordability both so that people are less likely to be homeless and so that people can do other thing with their money.
This is because homelessness is a side-effect of mental illness and drug addiction. The later is not all drug addiction, ( I used to live next to an "invisible" heroin addict which we only learned about when he OD'd) it's a certain type of disorganized-thinking and lifestyle which some addicts manifest.
The cost of housing doesn't cause homelessness; it causes roommates and the conflicts and violence and chaos that imposes on people's lives which is significant if under-studied.
The above assumes that you actually pay the house off of course. Many people don't.
It's impossible for a home to not be wealth, unless somehow infinite houses could be built for free.
So an argument can be had on how much wealth it should be and who gets to define that, but it is inescapable that it has some value.