Right, it seems like this only works if they conspire or if there are large property holders. Maybe it’s the latter? If I own two houses and left one empty it certainly wouldn’t change the total supply enough to comeback out ahead.
The real reason buildings would stay vacant is either ownership attempting to charge above market for rent, or land speculators who are looking to hold and flip, without developing it enough so it can be used in the meantime.