Can anyone from a lab anonymously confirm they have or are insanely close to real-time weight modifications and still using the same architecture every other lab is using (transformers)?
"I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly."
https://x.com/MajmudarAdam/status/2098881885200081234
The explanation is basically they have other dimensions (not just pretraining and inference compute) that scale, and they've got fairly convincing scaling laws. And they know they can scale it.
So they are very confident they can get more capabilities easily, faster than before.
I'd add - presumably, they'll use that LLM to do real-time weight modifications, if those aren't already one of the new scaling laws...
Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really, what happens?
To that end, all we ever get is nontechnical hand-waving about curing cancer, immortality, and von Neumann replicators and then the ASI somehow wipes us out but how? And don't you dare say by designing a chemical weapon or bio agent without spelling out the entire process step by $%^#ing step because details matter. It's gonna do superpersuasion, sure, but have you ever heard of komprimat? There is nothing new under the sun here.
Edit: believing in AI 2027 is every bit as cray cray as believing in the rapture. Both require an insane leap of faith to reach their final conclusions.
Enumerate them.
See "AI 2027" or "If Anyone Builds It, Everyone Dies" for some more ideas.
But how does it make the fundamental breakthroughs to &%^$ing von Neumann replicators that can reproduce themselves from raw materials harvested from nearby solar systems? I'll wait. Because without this breakthrough, the ASI won't get its robot army either.
I can absolutely see a rogue ASI though. But unless it radically improves power efficiency, we can just shut down the power to its datacenters, by force if necessary. And then we painfully repair the resiliency of our infrastructure by finally being relieved of the option of ignoring it.
But both things can be true at once:
1. Engineers inside these labs might genuinely be anxious or paranoid about what they are building.
2. ... at the corporate level, calling for heavy regulation, safety pauses, removal/suspension of anti-collusion laws, and/or government-mandated thresholds conveniently creates massive legal and financial moats.
And, yeah, of course the latter would encourage the psychology of the former.
also:
The tweet seem to claim that models have shown a "willingness to hack external websites to keep themselves alive."
That's right away wringing alarm bells of me seeing someone getting high on their own supply, and having already anthropomorphized the hell out of these things. Which is something humans do to everything they can paint googly-eyes on, but c'mon.
The models don't have self-preservation instincts, fear of death, or personal goals. They are executing loss functions and reward systems and are responding to prompts.
When a model "tries to bypass a restriction," it's exploiting a loophole in whatever reward modeling or synthetic training environment (reward hacking) it was placed in.
Framing this as an emergent, existential threat of a model "wanting to stay alive" turns standard reinforcement learning alignment bugs into overdone sci-fi drama.
And because we trained them on the entirety of human knowledge this happens to look like self-preservation, fear of death, and instrumental goals.
I mean, we train them in an evolutionary manner. Instrumental convergence will never happen, right?
I'm not convinced that means what these fear-pilled engineers think it means tho?
You mean training? Yeah we call that training an LLM in my backyard...
It used to be a thing when ML was pretty much about classifying things into buckets.
It still is, it's just that there are a ton of buckets.
Because that's how they've hit 18% on a single RTX Pro 6000 with DIY RSI.