This is why it is sometimes referred to as "location value" rather than land value. The dirt is approximately worthless, but the location is worth a lot due to the surroundings (usually, because of human development of the surroundings).

Advocates of a land value tax therefore often call it a location value tax, or a site value tax, because it is the location and the society around the location that has value, not the dirt itself.

Yes, that was my point. That is exactly how land appreciation works. The owner of the land gains value from the work of his neighbors, whether he contributes or not.