Based on my reading of Progress & Poverty, it’s not so much the finite nature of land that matters (though it does make the concern much more pressing). The reason land ought to be taxed differently is because the people who monopolize its productive power did not create the land nor its productive power.

The central thesis of Henry George is that a person is entitled to all value s/he creates. No one creates the value of natural resources like land (but also ocean, radio spectrum, etc.), but landlords get to monopolize its value. Since they have no claim to what they did not create, it would be economically just to tax that value and distribute it.