Whether it's Manhattan, Brooklyn, or even Paris... none of them would exist as we know them if we cared about "developments often ruin neighborhoods".
Neighborhoods change. What was once farm land and a few houses becomes suburban. What was once suburban becomes medium density. What was once medium density becomes dense city core. That's how cities have always worked. As the value of the land changes existing owners cash out and developers infill or rebuild. As density grows the justification (and budget) for city services like subways and parks grows.
Excessively restrictive zoning cuts off that natural process after the farmland-to-suburb step.
In free market terms if you cared so much about what your neighbor does with their own property you should have bought an interest in that property. Who are you to tell someone else they can't build a bigger building on property they own?
In reality there are few true free markets and externalities exist but in property rights terms zoning is a rather large government imposition to benefit other landowners often without any compensation to the owners who want to build. In an ideal world people who want to keep single family zoning should be paying land value fees to their neighbors who want to build density to compensate for the restriction. You couldn't possibly make such a scheme work in the real world though.