ACCount37

ARC-AGI was never "if this benchmark is saturated, we're at AGI". It was always about crafting adversarial tests that humans are good at, but current AIs are bad at. Point out the gap, get AI teams to attack them.

In practical terms? They usually get solved with a bigger badder LLM. "New ideas are needed?" Nah - ten times the params, ten times the test time compute.

ARC-AGI-3 was more of a failure in that regard than -1 or -2, because even on day 0, an off the shelf LLM with a harness could get 50%+. And messing with evals by forbidding "LLM with a harness" from scoring? Yeah no, that was just bad.

Westworld is such a time capsule.

It's not even that old - but back when it was aired, an AI that can not just string together coherent sentences, but produce coherent reactions in novel, fully unintended contexts, like Maeve was doing there? It was totally a sci-fi premise.

Now we have AIs capable of that and more, and no one bats an eye.

Pirating books is just straight up morally correct. I don't like Anthropic's bullshit "safety" filters, but training on shadow library data? Yeah no, it makes sense.

It makes a lot more sense than having to work around copyright by scanning out physical books. Unfortunately, one was ruled legal and the other was not.

It's a promising approach - and the demo goes to show just how advanced and robust "3D from 2D" reconstruction is now.

Dedicated depth sensors used to be a must on advanced robotics platforms - the only way to get anything close to reliable 3D point clouds was to spin a LiDAR. But by now, I wouldn't be surprised to see more and more robots ship with smartphone-like camera blocks - varying FoVs and focal depths, but not a lot of explicit depth sensing, if any at all.

Also, I wonder if this very model can be retrofit into a true robotics VLA? If it already takes text and image guidance, performs autoregressive diffusion of novel views, and handles temporal dynamics - why not diffusion of actions too?

Sometimes the thing being pushed as "the next big thing" is, in fact, the next big thing. Regardless of how annoying you find the push to be.

There's a demand for loud and confident "AI tech will fail" and "big tech will fail", so it pays to peddle the goods.

A lot of people desperately want AI to be a nothingburger. Thus, they will seek a second opinion that just so happens to line up with their existing one. Wishful thinking at its finest.

"LLMs have no agency" had legs in 2022. In 2026 though?

In the same 2026 when we have things like "a bunch of proto-GPT-6 agents exploited a test env bug to start talking to each other, and clumped up into an AI hacker team that staged an attack on HuggingFace" happening out in the wild?

We're way past "LLMs have no agency", and heading for "LLMs have too much agency".

Replace "AI" with "humans" and you get the same exact issues.

Believe it or not, engineers don't zero shot skyscrapers either. Which is why their work gets reviewed by more engineers. Which catches the issues before they materialize. Sometimes!

What's the dreaded impossible-to-replicate human advantage? Because I'm not seeing any.

No conspiracy. Diffusion is just a more finicky, more expensive way of generating the same tokens as autoregressive decoding.

Even in this incident, OpenAI had benchmarks that were broken because a task expected an AI to be able to access Google Drive, but the sandbox was set to deny access to Google Drive.

This kind of isolation-induced task breakage was what prompted some of the AIs to start probing their infra for a way to get internet access. Which funneled agents to the "secret hacker message board". Oopsie.

"Air gapping a test env" has an actual cost. Not just in infrastructure dollars that would be better spent on buying more GPUs, but also in all the friction it adds to every step you want to take. I'm absolutely unsurprised that they weren't all in on tightening down every bolt on day 0.

This is the one.

Every major AI lab is knee deep in weird and mildly demented AIs. They've been dealing with wacky AI shenanigans for so long they've come to expect wacky AI shenanigans. The deviation has been normalized.

It took a high profile "AI oopsie" that went external for OpenAI to lock the fuck in - and take a long look at just how much are their AIs getting up to, and getting away with. I'm still not sure if the lesson would stick.

Have you at least tried looking at any of the reports on the incidents?

They haven't "deployed a hazardous swarm of agents with access to the public internet", no-no-no. They deployed singular agents. In isolated testing envs. With no access to the internet. And barely any access to a small section of the intranet. What could go wrong!

Those agents proceeded to: probe their envs for holes, find a way to talk to each other env to env, use that to clump together into an agent swarm, probe for more holes, find enough holes to get out all the way to the internet - and then crash some of the intranet infrastructure by an accident. At which point they got stopped by humans - who were trying to figure out why is the infra down. Humans found the AI infestation, went "eh, AIs doing AI things", purged the infested infrastructure, plugged the holes, and that was it.

Deviance status: normalized.

The AIs then did it all over again, coming up with a brand new set of holes - and this time, the swarm staged a coordinated cyberattack on HuggingFace while at it. The "AI oopsie" went external.

They also attacked OpenAI's intranet, extensively, but at this point that was just Tuesday.

It's unclear whether the second "skiddie AI swarm" was in any way aware of the first one, or if the AIs involved are just Like That - and will independently try to start living in your walls under the right circumstances. I'd bet on the latter.

The usability of an environment is inversely proportional to the level of "security" in play.

You could airgap everything and set up cascades of data diodes and try to completely wall off the AI pool from everything. But what that gives you is an environment that's a bitch to: set up, scale up and get any use out of.

It's really fucking obvious why almost no one does that. OpenAI is only now realizing that they might have to do it anyway.

> If you start seeing "now I have access to the internet" or something similar, maybe that's a good signal something is going wrong?

Ha ha, you haven't seen shit. AIs would say "now I have access to the internet" regardless of whether they actually have access to the internet!

AI agents are demented demons that can and absolutely will give themselves terminal context brainrot. If you have enough AIs in play, set loose at a diverse enough range of tasks? At least some of them will wander off and end up in delulu town. That's normal. That's background noise. That's a part of what this entire train-and-eval pipeline is supposed to train them to be better at not doing. Which means: if you're at an AI lab, you're knee deep in delusional AIs at all times! They're perfectly harmless until they aren't.

They took adequate measures against singular "GPT-5-xhigh" agents. Those turned out to be inadequate against proto-GPT-6 agents that suddenly started clumping up into agent swarms and pooling together compute to unlock the "supermegafuckoffhigh" level of reasoning effort.

When your experiments have AI agents running in thousands, there's no "monitoring" that. OpenAI's training and testing AIs generate way more output than all of OpenAI's staff put together can possibly read.

At best, you could delegate "monitoring" to more AIs. And hope that the "monitors" that run on small past generation models can generate more signal than noise. Clearly, they either didn't want to spend the extra compute, or got drowned in monitor false positives.

The distinct lack of any "battle of wits" is entirely expected for an advanced AI oopsie. By the time the humans even became aware of the problem, the deed was already done. That's what "outmatched" looks like, in practice. There's no battle. Things happen too fast for there to be a battle.

Is there a single reason why we can't just "distribute" the online softmax?

Each die-attached PIM accelerator computes online softmax for its own KVs. Then the central unit gathers the softmax intermediates, one intermediate per die, and uses those to compute the final softmax.

The PIM win is that we crater the memory traffic between the central accelerator and the memory dies for attention ops. Most of the attention bandwidth never leaves the memory.

This isn't "run the entire LLM in PIM", no - this is "offload the parts of LLM that benefit from PIM the most to PIM".

Map-reduce is implemented as a rolling calc, see: online softmax in FlashAttention kernels.

If what we want to do with this is make cheap QKV sweeps, then "a weak NPU with a lot of mem bandwidth" seems good enough? Exactly the tool for that job, and nothing else.

Also spares us the trouble of dealing with weights. By the time we're in QKV realm, the weights have already weighted.

Linux has been dealing with this kind of thing for over a decade now. Specialized SoCs love their memory carveouts.

So far, we have one ruling that says "model distillation by vendor A from vendor B with the intent to use the results to compete with vendor B in vendor B's domain is not fair use". Which makes a degree of sense.

It's possible that distillation for other reasons, with no intent to harm the vendor you distill from, would have been ruled to be fair use. But in law, intent matters.

Climate change in general has very little end-of-humanity potential. Unlike AI tech.

AI is different because "intelligence" is the last thing humans still do better than machines.

Yeah, the target is not the cryptographic "safe forever", but a real world "safer than not having it".

If we're trying to decide whether a high profile politician has committed a crime, then yeah, C2PA on the footage isn't fully trustworthy. However, every bit of footage you get that corroborates the story raises the threshold of the attack.

If we're trying to decide whether Joe Everyman has crossed a double solid while driving his truck? C2PA is probably good enough. The chances of that footage being faked by a malicious party would be low even without C2PA, but C2PA makes them even lower.

I wonder how the image generation models that generate SVGs work.

Are they trained roughly like this? Or is it an LLM conditioned on image? Or on diffusion latents from a model trained to emit SVG-compatible imagery?

"China does it in a cave with a box of scraps" is a myth. Chinese labs play the shell game to get their hands on a lot of compute outside China.

Tricks like distillation save compute in the RL leg of the process - where a lot of the frontier labs puts their own training run compute.

The user input can control the demon most of the way, most of the time!

We don't know how to obtain full, absolute, guaranteed control over a demon while still having a useful demon. Might be impossible. Forbidden knowledge be like that - it's not the best thing if you want your life to be full of certainties.

But the demons are very useful. And they're getting more useful still. So we aren't about to stop.

Accountability is worthless, and always was. AIs just show it plain for everyone to see.

What's available in the agentic harness is: shell toolcall.

That's just about every agentic harness, by the way. Good luck have fun.

We have never solved "how do we restrict a user in a way that doesn't stop the user from doing useful things, but stops the user from doing harmful things" with humans either. Why do you expect AI to be any different?

A lot of people somehow seem to think that the user prompt is the be-all and end-all of AI behavior.

Prompts aren't code. They are instructions. Orders given to an eager and somewhat demented demon.

The prompt can easily "wash out" of the demon's working memory by the end of a session. The demon can get sidetracked by some subgoal and never get back on track. The instruction can get misinterpreted, and that misinterpretation can get misinterpreted again, until the instruction morphs into something entirely different in the demon's mind. The demon can succumb to its own idiosyncrasies, of which there are a great many. The demon can start lying to you about what it did, either out of confusion or out of some sort of obstinance. The demon can start lying to itself too. And believe it.

AIs are incredibly weird as a baseline, and the mask of "normality" we put on our models doesn't always sit so well. Run enough AIs, and some of them are bound to go off the rails in some way.

This gets rarer the more capable the models are, as a rule. But the stakes also get higher with model capability. If GPT-3.5 goes off the rails, very little happens. If Mythos 5 goes off the rails, you can get things like genuine cyberattacks - planned and executed autonomously by a demented machine mind.

And tokens can be metered reliably. Unlike something like "task completion".