You're correct, but the other thing with housing is that we actually have a supply glut. There are something like 16 million vacant homes in the US (according to numbers from an internet search).

Why so many vacant homes? The answer (as with everything in economics) is that the incentives are such that it's better to sit on a vacant home. In other words, our current system incentivizes hoarding homes.