Government already has the prerogative to set taxes, so you should worry about that exactly to the extent that you currently worry about it. LVT is orthogonal.

Broadly, government is not called rent-seeking because it does things to justify its keep: law, defense, roads, bins, etc etc.