Restricting home ownership as investment while not restricting apartment property investment or commercial property investment doesn't make things more fair.

It just puts property investment out of reach of the average middle class person and restricts social mobility.

e.g. a high-school drop-out plumber can afford to buy a house and rent it out based on just wages/credit, but he likely can't afford to do that with an apartment building.