My parents bought their house when rates were about 12-14%... "High" is relative.
Given a fiat currency and fractional reserve banking, the dollar floats in value like a leaf on the wind.
Houses cost less, people made proportionally less (inflation is a consideration here), and interest rates still bite you in the butt over the long haul, regardless of your monetary baseline.