If you buy a house to speculate on housing prices, and then rent it out, _you aren't adding anything_. That's entire point of the phrase "rent seeking". You've dumped capital into a limited-quantity good and then extract rent as it becomes more valuable.
The only exception is those actually expanding the housing supply (which is hugely valuable and should be rewarded!), but this is a tiny tiny fraction of the housing stock! Most commercial real-estate is not any housing-add over if they didn't exist!