> They need huge compute

My understanding is that huge compute is necessary to train but not to run the AI (that's why using LLMs is so cheap)

> To me the more real risk is creeping integration and reliance in everyday life until things become "too big to fail" so we can't pull the plug even if we wanted to

I agree with that, see e.g. what happened with attempts to restrict TikTok: https://en.wikipedia.org/wiki/Restrictions_on_TikTok_in_the_...

> But I would imagine if it really became a genuine existential threat we'd have to just do it

It's unclear to me that we would be able to. People would just say that it's science fiction, and that China will do it anyway, so we might as well enjoy the AI