I've designed a few homes and while zoning can be very constraining, it's not all bad.

When I do a daylight study, its usability in and around the home along with the impacts on energy efficiency are the top concerns when it comes to the design, but considering the neighbours is important too: In city lots at higher latitudes it's easy to cast a patio next door into permanent shade for large parts of the year.

For just this reason, I wouldn't throw away all zoning. Setbacks and height restrictions can really constrain me, but prevents me from being a jerk.

Density limits and a lot of occupancy type restrictions, on the other hand, can go to Hell. They prevent communities from having diverse properties (and the self sufficiency that comes with it) or evolving to meet local needs.