Having the population located in dense urban areas is more efficient and therefore cheaper in terms of the real resources that are required: infrastructure is cheaper to build, distances are smaller so that less time is wasted commuting and transport is cheaper (not to mention that highly efficient public transport becomes viable).
So in real resource terms, urban sprawl is more expensive.
But since land is still a limitation and zero-sum, competition can drive up housing costs in attractive dense urban areas to the point where it more than cancels out the efficiency gains of the urban density, and the cost of living in the urban area to the individual becomes higher than the sprawl.
This is why some sort of collective planning is required to make sure that as much as possible of the efficiency gains of urbanization are actually realized. This means having proper infrastructure mostly in terms of public transport.
(Of course, the other part of it is that you need to be careful not to look at just housing costs in the first place. For example, I don't even have to pay for a car, and I benefit from city infrastructure in many other ways.)
For the efficiency factor to overcome the desirability factor, we would have to actually issue the building permits to allow thriving urban areas to overtake some of their surrounding lower-density neighborhoods.
We've decided as a society that this is unacceptable, so urban areas will probably maintain their premium indefinitely. In a sense you're right. But only for self-inflicted public policy reasons.
I don’t own a car, offload a bunch of concerns into City infrastructure, walk to a corner grocery store, and go from idea to meetings inside a single day, thereby letting me dance circles around competitors who have to schedule a call or flight out.
Cities don’t make sense for everyone. But they do for a lot of people. They make so much sense for enough people, that those people will pay a premium to have everything else nearby.
A 40 minute walk and a 40 minute train ride are both reasonable commutes, but added together are nightmarish. I’ve tried.
i.e. staying within the radius you can traverse on foot. Our oldest villages may be small, but they are unthinkably dense compared to midcentury American suburbia.