This is the same intellectual laziness that says Muslims are terrorists, white people are slavers, and black people are criminals.

You are talking about the "mere addition paradox" [1] (aka the "repugnant conclusion" mentioned in the article). The point, as I understand it, is that whenever you maximize a metric it ceases to be a good metric, and optimizing for "people living" you reach a bunch of bad conclusions such as the one you mention where the more lives the better.

You say that you wouldn't do something like that, but the core of the question is "why?": what would be the utilitarian argument against going that way? You could argue that "happiness" should be a measure, at which point you need to contend with the problem of an "utility monster" [2]: what if it turns out that (say) Americans enjoy a dollar more than shell-shocked Ukranian refugees? Should we optimize the happiness of the Americans? How about those people with negative happiness, should we simply euthanize them?

The fault, according to the interview, is that you can't treat happiness and well-being as a matter of sums and averages. If you kill a man to open an ice-cream factory, no amount of happy children should absolve you of that crime.

[1] https://en.wikipedia.org/wiki/Mere_addition_paradox

[2] https://en.wikipedia.org/wiki/Utility_monster

There isn't One True Utilitarianism anyway because you still have to specify a utility function and the aggregation of it that you want to maximize. And that's an AGI-hard problem. So in practice you can only do vastly simplified approximations of utility and then occasionally check whether the outcomes make sense in an intuitive way.

https://twitter.com/esyudkowsky/status/1497157447219232768

> Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the right place. Stay there at least until you have become a god.

You might need to read up on the repugnant inclusion. It compares a small population of high individual welfare with a large population of lower welfare. If you assume the desirability of each option to be the product of individual welfare and population size, each population of high welfare has an equally appealing alternative population of many people living just above the point of not wanting to exist.

Utilitarianism requires additional assumptions to decide between many people in subsistence and few people living carefree.

You are introducing (I suspect) such an addictional constraint, comparable to the Pareto criterion that forbids making individuals worse off in a trade off. However, accepting/demanding/proscribing such trade offs that accept making someone worse off are central to most utilitarian theories.