This is the key line "the fundamental tension [is] that Americans want housing to be both affordable and a good investment."
You can't have both! You cannot have your home be a good investment because prices keep rising and have homes be affordable. We have a national narrative that the best path to wealth accumulation is through home ownership. That is a terrible terrible narrative to promote. You can make it true, but only if you massively subsidize home ownership (which we do) and make in incredibly difficult to build homes in desirable locations like most of California (yup we do that too).
We need a new national narrative. Housing is not a tool for wealth accumulation. A home is a place to live. The cheaper that place is the more people will be able to live in one.