Of course, if you live in California or New York, you're basically broke, since the cost of living in those states is designed to keep people poor. But that's why people left those two states in droves over the last few years.
I'm from a working class family myself. My parents were factory workers, and they managed to buy a new 4 bed apartment in a new part of town and raise 3 children.
And yet I don't feel I'm doing materially better than my parents at my age. I definitely don't feel upper middle class even if my income suggest so. As you point out I can't imagine how people with more average incomes get by.
I would venture to say that more land area of California is in the "unaffordable to average people" range than that in New York, either as an absolute or as a percentage of the state.