So when you say landlords are the least productive class in society, it is that they are totally unproductive by their nature. There is literally nothing to be produced! The land is there regardless! Then the fact that they collect a tax on consumption of a good they did not create makes them a negative-productivity component.
If you remove landlords (but keep property managers, developers, etc, which are all different jobs sometimes played by the same people), it would be a strict improvement to everything about our economy.