Drinking a glass of water is beneficial and necessary. Drinking 100L in one go and you're dead.
The argument is that the rate of AI influx is too high for our human systems to metabolize, and so the system will break down. Not just operate in a degraded way, but break down and cause mass catastrophe. Even there, it doesn't necessarily mean the end of humanity, just 99.9% of us.
What the argument is not: AI is fundamentally poisonous. We are incapable of handling any level of AI. We will never be able to incorporate more AI. AI provides no benefit.
So quitting AI but still working on AI is not the contradiction it seems: you can quit accelerating AI and work on metabolizing it. And in fact that work is very necessary right now.