The opinions seem to fall into two camps:
1) This is just a move that evil tech companies are making in order to control who has access to AI and to maintain their dominance
2) AI is scary af and we are at a inflection point in history where we need to proceed cautiously.
This NYTimes piece is clearly debating the latter point.
> artificial intelligence technology [that tech companies] are building may one day pose an existential threat to humanity and should be considered a societal risk on par with pandemics and nuclear wars.
To people in the first camp of thinking this may feel like an over-dramatization. But allow me to elaborate. Forget about ChatGPT, Bard, Copilot, etc for a second because those aren't even "true" AI anyway. They simply represent the beginning of this journey towards true AI. Now imagine the end-game, 30 years from now with true AI at our disposal. Don't worry about how it works, just that it does and what it would mean. For perspective, the internet is only about 30 years old (depending on how you count) and it really is only about 20 years old in terms of common household usage. Think about the first time you bought something online compared to now. Imagine the power that you felt the first time you shared an email. Then eventually you could share an entire photo, and now sending multi-hour long diatribes of 4K video are trivial and in the hands of anybody. That was only about 20-30 years. The speed of AI will be 100x+ faster because we already have the backbone of fiber internet, web technologies, smartphones, etc which we had to build from scratch last time we had a pivotal technological renaissance.
It is easy to shrug off rogue-AI systems as "science fiction", but these are legitimate concerns when you fast forward through a decade or more of AI research and advancement. It might seem overly dramatic to fear that AI is controlling or dictating human actions, but there are legitimate and realistic evolutionary paths that take us to that point. AI eventually consuming many or most human jobs does in fact place an existential risk on humanity. The battle for superior AI is in fact as powerful as the threat of nuclear weapons being potentially unleashed on an unruly country at any time.
ChatGPT does not put us at risk of any of these things right now, but it does represent the largest advancement we have seen towards the goal of true AI that we have yet to see. Over the next 12-18 months we likely will start to see the emergence of early AI systems which will start to compound upon themselves (potentially even building themselves) at rates that make the internet look like the stone age.
Given the magnitude of the consequences (listed above), it is worth true consideration and not just shrugging off that I see in many of these comments. That is not to suggest that we stop developing AI, but that we do consider these potential outcomes before proceeding forward. This is a genie that you can't put back in the bottle.
Now who should control this power? Should it be governments, tech companies? I don't know. There is no good answer to that question and it will take creative solutions to figure it out. However, we can't have those discussions until everyone agrees that if done incorrectly, AI does pose a serious risk to humanity, that is likely irreversible.