New York has a legitimate demand problem, seeing as it's the capital of the world.
California has an artificial demand problem due to NIMBYs who corrupt zoning boards, oppose reasonable development at every turn, and live in the past while paying lip service to "building the future".
(It also has a problem with corrupt/dirty foreign money swelling into the housing market, but that's an issue in New York as well.)
New York has solid public transportation. You don't need a car if you live there. People bitch about the MTA, but it's remarkably cheap compared to having a car anywhere in the US, and remarkably convenient considering how much access it provides.
San Francisco is a city where you need a car, but can't afford it. Parking costs are so bad that many garages post their 15 minute rates (often around $3.00).
New York is hard-core urban and owns the positives and negatives that comes from it, and New York runs itself. San Francisco companies are run from the suburbs-- specifically, from an office park called "Sand Hill Road" in a stretch of land that really ought to be reclaimed by the cows and orange trees, because the people living there are contributing nothing but additional work for the sewer system.
Manhattan is what it is, take it or leave it. (And there's nothing wrong with leaving it.) California is fundamentally dishonest and hypocritical, because it's full of people who pretend to be liberal but are, in many ways, worse than the Manhattan old board when it comes to elitism.
New York isn't perfect. There's a lot that's badly designed or unpleasant about it, but if you compare it to the cataclysmic mismanagement of California at all levels, it's a breath of fresh air.