Every zoning wonk's favorite type of zoning, progressive zoning (i.e. "the kind of zoning Japan has") specifies residential as the "base" type of zone, that all other zones must also allow for. So there's "residential", "residential + commercial", "residential + commercial + light industrial", etc. I've never seen this explicitly stated, but I would assume that this means that all types of zoned land must have their public infrastructure done up to the standards required to build residences on them. (Not necessarily dense residences, but residences nevertheless.)

However, that aside, I don't think the GP is suggesting rezoning office towers in a downtown core such that they get torn down and replaced with equally-dense condo towers. I think the much more common use-case that everyone wants to see, is taking the land on which sits e.g. an abandoned mall, or a disused business park, or something else that's not already surrounded by hard-to-work-around density; rezoning that land to mixed-use commercial+residential; and then selling the land to a property developer (who then incurs the costs of infrastructure building) to blow it away and replace it with a "village" of midrises + shopping laid out for walkability.

Or, alternately, we can approach things from the other angle: upzone light SFH-only residential, into mixed-use medium-density residential + commercial. I.e. tell property developers that they're free to buy up stretches of homes and plop down 5+1 apartment buildings with shops on the ground floor; and also allow anyone living in a home in the area to run a corner store, bistro, dance studio, hair salon, or other (non-industrial) businesses they like out of their house, with no need to apply for a zoning permit.

In short: why try to cram more people into downtown, when you can instead spread out the benefits of living downtown?