ctolsen

Opus 5.5: $4/$20

Deepseek 4.1: $0.02/$0.60

Just to illustrate how cheap the Corolla is in your analogy. Also Opus output would be $50 without competition.

Depends what you do. I enjoyn hosting big parties because I spend a lot of time cooking for them, which I love doing and everyone appreciates. Then I have the opportunity to interact as much as I want.

Agreed, and if you look at the public information about various attacks the individual vectors are not particurarly sophisticated. The Huggingface incident for example had agents in a sandbox that was about as useful as a wet paper bag, and the attacks on remote infrastructure were basically enabled by bad sanitation. None of the techniques are novel.

These are not hard problems to fix, nor should anyone find it acceptable to have them be so prevalent. A determined human attacker could easily exploit defects like the agents found. Bad input sanitation, SSRF attacks, exploiting stupidly implemented token verification... all techniques that have been widely known for decades and there should be no excuse for publishing software that is riddled with exploits that enable the use of them.

> My update for you since then: even an air-gap will be inadequate, there's multiple known ways around them.

Fair, and I will grant that a capable model (or human) could in theory break out of near anything.

My point is that this incident is not evidence of that. There is zero skill visible in the setup of the sandbox. Nobody messed up a critical detail, they didn’t even start to consider what the details were.

I doubt most people "blind to the possibility" would imagine that what we’re measuring against is the equivalent of benchmarking burglar skill based on how easily they can break through an unlocked door.

> Now we know that.

Don’t know about you but it’s pretty obvious to me that you would need more than what OpenAI did. It was not remotely adequate to lock in even a human attacker.

You can find people who say all sorts on the internet, but this case is not much evidence against what you linked. "Zero-day" makes it sound novel, but the breakout patterns here are based on very common exploits and there’ll be plenty of examples in training data.

Part of the toolchain had full internet access. Agents had access to Artifactory, Artifactory could do whatever it wanted. So even locking that down to certain external sites would have stopped this particular attack.

Don’t let software inside the sandbox access the internet on request. Have a package repository with approved software the agents might want, and push to it over a channel that is ingress only. I can imagine ways of breaking out of this, and come up with a lot more to mitigate, but this would be fairly basic stuff that’d be vastly superior.

Right, that’s basically what I’m saying. There’s not zero use for an attack like this, but it’s not a likely situation.

Said another way: if Artifactory was somehow a common gatekeeper between grounded techy teenagers and their access to internet porn, this would have been found ages ago.

Why would it be getting attention? There’s an enormous amount of garbage software out there, and not an endless supply of researchers. Especially tooling like this, used internally where it’s assumed that security is the job of something else. It’s far from the first time serious but simple vulnerabilities have gone unnoticed for a long time. And the use case of having this be your way to the internet is probably rare, so nobody has tried very hard (or tried but never told anyone).

I’ve looked at the CVEs a bit more and it’s just very clearly a pattern of systemic issues with validation, be it URLs or tokens.

It just shouldn’t be that hard to believe that OpenAI just didn’t care very much and thus did a crap job. The whole model of the sandbox is terrible, so why would they bother thinking about the implementation much?

So like I said: unscrutinised and riddled with issues.

It’s very much solvable, they just don’t care.

> Are "half skilled human operators" "easily" able to find zero-day vulnerabilities in a sandbox with only one line to the internet (the commercial package registry cache proxy)?

Yes. It’s a fairly simple SSRF attack as far as I can tell. One of the first things I’d try. Especially considering that I would already be armed with the information that I have no internet access except through a thing that downloads things off the internet for me.

Calling it zero day makes it sound elusive. It’s a bug in closed software that has like 40 CVEs this year alone. Tools like that, especially in internal networks, don’t get much scrutiny and are often riddled with issues.

My biggest takeaway from this is just how godawful the sandboxing is. The stuff written up in OpenAIs report says more about lack of extremely basic sysadmin skills than anything else.

I’m not that surprised about models with endless compute being capable of this, I’m more surprised that a company with the resources they have apparently can only create a sandbox that a half skilled human operator could have broken out of easily.

I particularly enjoy the Asterix-adjacent naming scheme.

And the local drug dealer says he's a sovcit but is still somehow in jail. What's your point?

The ICC claims rights in territories where the sovereign power in that territory has signed and incorporated a treaty that grants those rights. ICC jurisdiction is domestic law.

If that's a talking point it's a stupid one. The Rome Statutes are only valid in territories that have agreed to be bound by them.

Definitely a little better.

https://gist.github.com/ctolsen/b2883e7cbf5e4357fa04366019e6...

If we go with AA's benchmarks Qwen 3.8 27B is already slightly below Luna level which is in itself impressive, but with this compression it should be just slightly more below Luna level and could run on my old GTX 1070 that I'm now tempted to fire up. That's kinda nuts even allowing for small-model problems that I'm sure I'd see quite clearly.

Their ambition isn't your work being amplified by their model, they want you running fifty autonomous long-running agents.

The systems that train them do.

So few terms in AI are well defined. We will get ASI via AGI because of RSI but neither of those three things have any definition except pure vibes.

I struggle with the argument that RSI doesn't already exist like you say, it's existed since before the term LLM (hey, one that can be defined!) was common parlance. Though the biggest use for those is not superintelligence, it's to serve you ads and get your kids addicted to TikTok.

You gotta keep up. Fable 5.1 came out yesterday and is better so anything else is to be treated as garbage now.

I'd go for the less complex version: just make copyright last for like 20 years at most.

No, it’s definitely not what a human brain is. That makes very little sense. The ways we interact with language (and thus conceptual memory) is completely and fundamentally different.

Yeah, if this is stopped, it'll be because of the European Charter of Fundamental Rights or the ECHR.

The Charter and the European Court of Justice is why we don't have blanket data retention in the EU but it took twelve years to strike down the Data Retention Directive (though it was killed off much faster in some national courts).

It's well established that the companies who own the proprietary frontier models complain loudly that open models are distilled from theirs.

There's surely some truth to it (and it's well deserved), but it's happening in every direction.

I mean, I could say the same about Gemini. 3.1 Pro tops a bunch of benchmarks out there but any practical use I've put it to it's underperforming both other proprietary and open weight models. Benchmarks are suspicious in general.

Having worked with both proprietary and open weight SOTA models lately, my view is it's definitely not 6 months, it's less -- and shrinking.

It's 100% laziness on the side of procurement, aided by some good marketing and a complete lack of guardrails. Exactly the same mindset that has led to every European government now being tied to US big tech.