I believe the template for setbacks and height limits came from New York City, wherein a new 40-story building cast a good deal of the surrounding neighborhood into permanent shadow.
As the "Ancient Lights" doctrine of English property law had been repudiated in America, and the Prescription Act 1832 obviously came after independence, the neighborhood incumbents had no real legal grounds for complaint. This disturbed enough people that New York passed its first architectural restriction laws.
Land-use restrictions basically came out of racists trying to legislate segregation. Those nuisance laws combined with architecture laws to birth modern zoning. Once that camel's nose entered the tent, the rest of the camel came right in. Every problem that has a nexus with private land use had an attempted zoning solution attempted somewhere. So things like parking-space allotments and environmental impact assessments were added. The NIMBYs could halt developments they didn't like by dealing the death of a thousand permits and public hearings through the zoning laws.
I think there was a law in Boston regarding buildings that cast shadows onto public parks that affected the height of a building quite some distance away, as it would have cast less than an hour of shadow on a few autumn days, on one corner of the park.
Setbacks and height limits have always been about the fear of turning the city into a maze of dark corridors, bounded by high walls. The zoning planners wanted sunlight to fall upon the streets.
They may have overestimated the public's lust for light. A lot of us prefer that everything we wish to reach be within walking distance of our homes, and there are vitamin D pills to be swallowed and SAD lightboxes to stare into, to make up for the loss.
It's a similar argument as to why some people who live in the desert are not allowed to collect the rainwater that falls upon their property. Apparently sometimes you can own the weather, and sometimes it's a commons, and sometimes it belongs to someone else, somewhere downstream.
Was this a rhetorical question? "How does having my house near the street on my lot restrict public usage of the street?"
If you have heard the argument before, ad infinitum, then you obviously already knew one of the possible answers. So yes, it must have a rhetorical question. But no one else but you would have been able to know that, and I mistook it for genuine confusion or curiosity. Why would you criticize someone trying to answer a question that you asked? It would have been easy enough to mention that you had already heard the incident sunlight argument and reject it.
For the record, I reject it, but whether I agree or not, that is one basis for the architectural restrictions included in zoning laws all across the US. Knowing that it is such a trivial and frivolous reason is just another reason to dislike zoning, along with knowing that land-usage restrictions are based on racism.
I'm criticizing your unnecessary verbosity.