Imagine that there's somebody who wants to build unpleasant things. Let's say there's a group "association of people for ugly and genuinely unpleasant buildings" who build their collections of ugly buildings, buying up the cheap land that becomes available as people who no longer want to live near them move out.
Well, that group is rewarded by this system.
I think the problem is really: LVT makes sense, there's no reason to tax buildings, but if it is to be fair, it must actually tax the unimproved land, and this means it can't punish people who buy things next to someone building something attractive and then not doing anything useful with it, so that particular goal, sometimes held up as one of the points of a pure LVT is not achieved.