So what makes converting existing structures to residential use harder than building new residential buildings on empty land?

I view this as an "outrunning the bear" situation. I.e. you don't need to run faster than the bear, just run faster than your poor friend.

The cost to convert an office tower into apartments is high I'm sure, but is it higher than building a similarly-sized apartment tower from scratch? A lot of money goes into the foundation works and soils engineering.

And it can rather easily be added on the outside in some isolated ducts. The problem is cutting it down to appartments that correspond to fire safety codes. CAL demands you have two exits, which a office building easily fullfills with two staircases. But how do you do that in a converted apartment building, without loosing tremnedous amounts of space to hallways.

Everyone said the same thing about lofts/industrial spaces. Going by other contexts, the greatest profits will flow to people who Just Do It and make people happy, later seeking forgiveness instead of permission.

So make that kind of residential space... different.