If money was hard to come by, houses would be cheaper, but then you would also have more difficulty getting that money to start with. I'm not sure the affordability would improve, it's hard to say what other impacts a higher interest rate (or even more so, a higher % down requirement) market would create.
The only exception to all this are new houses and condo builds, but in those markets the reason they're more affordable comes down to a combination of location and risk, risk that the property may not actually be worth its selling price, so you get it at a 'discount' to the market if it were in a more established area.