Except that it's going to be robot police. What in the current world order suggests labor wins. Who controls LLMs? Who has the compute to produce AGI?
Not saying there aren't good points. Only pointing out that many, many people can all view the consequences of this dynamic differently.
I am replying to the parent comment, but I too get that this petition is being signed onto by people at openAI specifically because they already made it with GPT-4. They definitely would not sign on to it if their (arbitrary) threshold for what to stop doing AI research with was a tool at a GPT-3 or GPT-3.5 level or higher or it included GPT-4 in their threshold, and not merely beyond it.