It's not the land that's valuable but the lands proximity to goods and services - in that sense, it's not fair that the value created by those goods and services is accrued by a private property that did not pay a proportionate fraction of the revenue sustaining it as the bulk of revenue is raise from other means.

This is where the Georgist idea of land value tax comes in.