The real reasons are 1) architects more interested in getting applause from architects for cutting-edge design instead of utilizing tried-and-true principles for humane spaces, and 2) these things are planned, designed, and built by people who will never live or work in them.
Architects design for coffee table books and developers+planners design for spreadsheets.
I assume the real reason relates to "abuse of the commons" and the American problem of "why we can't have nice things" outside of "gated" communities.
Despite being farther from Cambridge (a different city than Boston, so they have a different mayor), I prioritize biking to places there to meet friends or go out to dinner. It is just a much more pleasant experience than nearly anywhere in Boston proper.
[1]: https://www.nytimes.com/2025/02/04/us/boston-mayor-josh-kraf...
These same people are ostensibly liberal, "we're all for people riding bikes, or getting a stupid little condo, whatever you kids like to do for fun" they say, but in practice will do anything in their power to prevent any impediment to their car commute, even if their fear is at-best specious.
Same thing in Toronto. In Vancouver Canada we're a fair bit better about the bike lane thing, though it's still fragile, but in the affordable housing issue they actually just subsidized the huge home owners that already owned land, so a majority of new units that have come on the market are in their basements or backyard and can't be owned, furthering a class divide. They marginally upzoned, but now a duplex or small townhouse is over a million at the low end. There isn't an incentive to make housing less expensive except the vague sense that the municipal/regional economy will eventually collapse on someone else's watch.
Whatever you think about all those issues, and how stupid you think the hoi polloi are, there has been a massive strategic miscalculation about which issues to focus on, and how to address them over the last number of years.