Think of it as a tax just like on land, but for buildings.

The reasoning is that housing represents an asset that should not be bought just for speculation, i.e. buy it and let it sit empty until the price is high enough for the investor to make a good profit. Of course they're free to do just that, but all housing which people can live in, are taxed as if you had that income.