"Producing something on it" is a wholly subjective standard. I can just as well proclaim the opposite, that not producing anything on it is the standard that should determine ownership.
Turns out… it’s still right there!
the georgist's view is that the landowner does little to contribute to the area being nice/successful, but reap all the benefits.
> Take now... some hard-headed business man, who has no theories, but knows how to make money. Say to him: "Here is a little village; in ten years it will be a great city—in ten years the railroad will have taken the place of the stage coach, the electric light of the candle; it will abound with all the machinery and improvements that so enormously multiply the effective power of labor. Will in ten years, interest be any higher?" He will tell you, "No!" "Will the wages of the common labor be any higher...?" He will tell you, "No the wages of common labor will not be any higher..." "What, then, will be higher?" "Rent, the value of land. Go, get yourself a piece of ground, and hold possession."
> [...] without doing one stroke of work, without adding one iota of wealth to the community, in ten years you will be rich! In the new city you may have a luxurious mansion, but among its public buildings will be an almshouse.
Unless there's a Khmer rouge regime in America that forces the intellectuals to move to the countryside at gun point most of the country will remain empty.
Expensive housing in many areas is a result of laws that keep would-be developers from increasing the supply, fueled by the toxic NIMBY mindset.