[flagged]
You are paying the government rent in perpetuity for the services they provide, including enforcement of your property rights. If you wish to live on land without paying government rent, you will need to go to a place without effective government. Today, that would be Syria, parts of Sudan, or Mali as examples. I doubt you want to live in those places as a 1st world resident, but if you do, there's little stopping you.
If you're talking about an individual not using the land commercially - well theoretically the tax burden lowers the sale price. You don't pay both. At 100% land tax, the sale price would be zero. You pay only as you use the land. That also seems just to me.
How much of the land value of your country do you currently own? A land value tax is likely to increase the amount of land you own (the profits of). I don't think US tracks land values as accurately as countries with some kind of land tax.