It's weights. Spun up billions of times over.
If humans are taking the outputs as scripture? As "intelligence we must obey"? Yeah, we're in for a bad time. "Intelligence we want running core infrastructure?" Puke.
But why would we do that? Why would we hook up an LLM to a robot factory, or make an ASI that could easily move itself into the ether?
Everything magic about LLMs is what harness we put them in and let them drive. There's nothing dangerous about predicting the next token. There's something dangerous about letting them do dangerous things, but nobody seems to care for maybe Tesla FSD. We don't think it's about to take over.
Any assignment of motive or whatever to the AI, you're talking about one cobbled-together harness or something, with an LLM manning the controls. You can swap the brain out from Claude to Astra and the motivated-AI will maintain its motivation--it lives outside of the inference or the data center. So yeah, let's not let it take control of the nukes. That would be really dangerous. But it's not going to take control on its own. "It" is a calculator.
Anthropic obviously has guardrails to prevent people from using AI to do all kinds of things already. All I hear about is you guys bitching it won't do your barely-related-to-security work. That's all we need to continue doing to have AI safety. Stop humans from doing evil and stupid things with it. The idea that the datacenter is going to assemble its own sentience and motivations is pure retardium (at this time, for llms).