The hugging face is hardly a warning shot of anything close to that scale. For now, I don't see (not saying there's not, I'm trying to also understand why these smart people think like that) any indication that the current development of AI will be as dangerous as some describe.
I understand the idea that if we let it control nukes, if we let it fully decides if a med is safe, etc. It poses a huge risk. But honestly I really don't see indications that it's going majorly this way, nor that it has anywhere the capacity to do something that big. It found a zero day and hacked a niche website, ok, so what?
This feels like a diversion from the actual existing and proven problem of all the problematic practices it's actually built on.
I would adhere way more to "we need to pace Ai because it could be catastrophic for society (jobs, cognition, loneliness,...) and environment" than "it has the potential to kill us all"