Down payments are the monkey wrench here. The requirement to have a down payment means high prices basically lock people out of homeownership, period. That in turn limits ability to build wealth at all, even at a crappy interest rate.
My experience from buying a house two years ago is that though I suspected it, I never truly realized how many amenities were covered by my rent. I went from paying $30/month in utilities, etc. to ~$400 /month. I also now live 10 miles further away from where I work / hang out.
Granted, we live in a house that is ~2x as big as our apartment, plus two car garage. But if you factored out strictly the various recurring expenses I now pay from my previous rent, it is extremely close to breaking even. And that isn't including the increased cost of owning / operating two cars that now drive ~60 more miles a day between the two of them.
Our net worth increased much faster while we were renting than now that we have a house.