"Just build tall buildings" is a necessary but insufficient condition for an effective city. Manhattan works not just because it builds up (in fact many parts of it are limited to 6-story buildings) but because it has mastered the complexities of creating livable, workable spaces.
The other downtowns across America saw Manhattan, and took only "build tall buildings" away as a lesson, and they are for the most part deserted, ill-occupied spaces. Simple density and height is not the most important factor - it's not even among the top-10. "Manhattanization" became a dirty word across the country not because Manhattan is bad, but because so many places copied it thoughtlessly and incompetently.
A study of what makes New York City tick that examines Midtown over long-standing neighborhoods like Greenwich Village, the Upper West Side, and even more outlying ones like Harlem and Washington Heights, is doomed to fail, because you're seeing a complex, functional city and the only lesson you're seemingly taking away is "density = good", despite nearly every other urban center in the US being evidence otherwise.
tl;dr: Density is a necessary component of a functional city, but you don't need Midtown Manhattan-level density, and having density is not enough. High-density failed urban spaces are a dime a dozen.
[edit] Side note: elevators aren't actually very efficient at high density. They are great for low-mid buildings because they add a lot of floor area and don't occupy much space themselves.
For very tall skyscrapers it inverts - the number of elevators needed to properly serve the building starts increasing dramatically, eating up valuable floor space. There is a level at which you lose overall floor space for each floor you add to the building.
Some of the supertall skyscrapers currently in existence/being built are in fact taller than they economically should be - and this is subsidized by pride, ego, government, rich benefactors, or some combination of the above.
Tall skyscrapers make all kinds of optimizations to minimize this - destination-based elevators (where you punch in your destination before getting in) are popular, as are express elevators segmenting certain floors, but all of these grant modest gains in elevator capacity at best, and in modern times skyscraper heights are limited not by materials or architectural engineering but by this function of elevators vs. floor space.