Sure, your home price would be reduced if somebody could build a pig farm, or a heavy-metal processing plant next to your house.
And it's not just an advantage for people who already own houses. Imagine how much of a down-payment you'd have to have--and how expensive mortgages would be--if the banker had no assurances that the value of your house wouldn't suddenly plunge to zero.
People can't afford houses because they are not making enough money. The solution isn't to make their biggest asset less valuable. And the solution isn't to make owning such a large asset even more risky to own.