Have you seen the Center for AI Safety letter? A lot of experts are worried AI safety could be an x-risk:

https://www.safe.ai/statement-on-ai-risk