Living in the middle of a dense city does improve quality of life for many people via access to amenities and jobs with a very short commute. A hypothetical 50 story skyscraper can provide these benefits to 50 times as many people as a single story building could.

You're right that light and sky visibility are finite resources that require more land. 100+ story buildings can feel oppressive to people. The question to ask then is why we allocate so little land to mid-rise development and force it all into Manhattan and small downtowns. That's the biggest reason we have such tall buildings there.