Home ownership in the US is around 65% [0], so the majority do have an asset and mortgage. There are also some people who choose not to own a house but have other types of assets. Things really aren't as dire as you seem to think.
And for those people who actually are harmed by inflation, I think the consensus is that they would be harmed by the other option--deflation--even more, mostly because the disincentive to invest leads to fewer employment opportunities. The Great Depression, a deflationary environment, was not good times.
[0] https://en.wikipedia.org/wiki/Home-ownership_in_the_United_S...