A large influx of units devalues similar homes in the same general location. People don't want their home price to drop. They may not have bought it specifically as an investment, but if you buy something for $500k and 10 years later it is worth $400k, you're not going to be happy. You might even go underwater on a mortgage because you only put 3% down for your first home.
I'm not saying it's right, but I do understand why NIMBY syndrome exists. People would be fine with more housing, more skyscrapers, or whatever if their home price wasn't directly affected.