High rates don’t impact people without mortgages. There are lots of people who bought in ages ago and move their equity between homes.
At least according to the WSJ and others, there seems to be a decent chunk of investor-owned real estate. Someone probably has a study comparing real estate in countries that restrict ownership by investors to see how much this distorts prices.