So if you buy a fixer upper, one that would be unfit for renting to people. You can't spend your weekends and holidays working on it for a few months to get it into better condition without being called immoral? Sorry that my lack of funds and desire to better myself offends you, but that's a crazy take.
Because our society believes in property rights. Individuals can have any reason they want to own property and do with it what it pleases them.