Imagine wanting to start an urban community garden and make your city a nicer place to live, but the taxes are prohibitive because you could have built a concrete monstrosity there instead.
Taxes predicated on the money you could have hypothetically made by paving over everything is a great way to ensure that you'll pave over everything.
Property taxes in many places currently tax improvements, so if that concrete monstrosity wanted to build a nice community garden on the premises it would often result in a tax bump. That typically isn't the case with LTV.
In contrast, if the land owners surrounding the concrete monstrosity build higher density housing with gardens and public areas then they may drive up the LTV such that the concrete monstrosity is no longer economically viable and must be torn down and updated.
I'm sure we all look forward to a world in which buying hookers for the Mayor's son and keeping it all on the DL, or contributing to a few Town Selectmen's "nonprofits" is what you have to do to in order to inject a half-acre of greenery into an otherwise desolate dystopian concrete hellscape.
But yes, that is a consideration that must be made. Georgism is a very logical, mathematical train of thought, and as such does not effectively (in isolation) address human subjective needs, which certainly includes living in beautiful (or at least, non-terrible) places.