You mean if you own a massive complex, which is the vast minority of housing - and also not what was mostly accused of being bought up by individuals or firms for speculation.
If you're talking smaller units, landlords are effectively price takers and it is pretty much never profitable to keep empty and price up everything else unless you are anticipating immediate increase in prices.
None of this is a major driver of housing prices either way.