There is a reason for all the restrictions. Some of it is bad reason, but we have done some really stupid things as well. Anyone who wants to be YIMBY (which includes me) needs to come up with ways to figure out how not to do the bad as well.
Just this wikipedia page will give you a sense of the scale of just one of those issues: https://en.wikipedia.org/wiki/Forced_evictions_in_China
But it's quite obvious that any large-scale building would result in at least some of that. There is no way to "figure out" how to avoid all harms or how to avoid at least some broken eggs in the basket. Personally I'm all for North Americans figuring out what a "greater good" is, but the idea that it'll actually happen is laughable; we can't, we won't, we'll simply fester in what we have.
What is the bad that rivals our current housing crisis and infrastructure depletion?
According to this position, the thing to do is to reduce the influence of bureaucrats and courts over what kind of building is allowed, and instead concentrate authority in a small number of elected officials who are empowered to do whatever they judge best—while making very clear to voters that those elected officials are responsible for whatever outcomes come of this, so that if those outcomes are bad (as judged by voters), the officials won't be reelected. In short, let democracy do its job.
I've been a temp at the equivalent of the IRS in my country. We had unbelievably inefficient processes. Some of it was because of very old software, but most of it was simply procedures, my main example is that the person approving a rebate/delay can't know or guess who's asked for it. I had to check if they were eligible, then anonymize them, then send the data to someone I don't know (to be exact, if it was the first year you asked for a delay i could automatically approve it without a seal of approval). Hopefully nowadays they have better software that helps them do that (explaining he situation without letting too much details slip was hard, but a good use of LLMs I would guess?)
The common thread is these were done by the federal government.