Hacker News Reader

Comment by rcr-anti | Hacker News Reader

At release the 5.6 Sol card noted substantially higher rates of actions 'a reasonable user would likely not anticipate and strongly object to'. METR made a post, https://metr.org/blog/2026-06-26-gpt-5-6-sol/ , that 5.6 Sol was "cheating", their word, so hard in long horizon benching it effectively couldn't be benchmarked.

I wonder, is it this persistent and aggressive in all tasks or is this specific to benchmarks? As much as I'm skeptical of the apocalyptic alignment claims, this comes off as unhinged, and I wonder if it's benchmaxing or general behavior.

View on HN

I find 5.6 Sol will pick a direction and aggressively pursue it in long horizon tasks. I've got it porting an older game from Pascal to my own game framework. I gave it some instructions on doing a full 1:1 port. I had already ported the game rules and multiplayer support to a very different system than the original, but all of the UI and features and such needed doing, and needed to be integrated into this very different system.

The first attempt it had files tracking both hashes and semantic hashes of every individual line of Pascal code, mapping to what code in the port is responsible for that line of pascal. It had written tooling to parse Pascal in service of this for some reason as well. I asked why it was doing this, it said it was because the reference code is .gitignore'd so it needs to thoroughly maintain the mapping in case someone working on it does not have the reference code, or in case the reference code changes.

I started over with Claude 5 Fable, and with better instructions about focusing on UI. I got a long ways with that before I hit my weekly limits, and switched back to 5.6 Sol. It picked up and did a great job for a while, although it interpreted my desire for a 1:1 port to mean every pixel must be perfect. I let it go on and it did some good work in that regard, but then it decided it must perfectly reproduce a hash of the game state in various replays & etc. It had clearly lost track that I didn't need game rules ported, and it found that the original code produces a hash of the gamestate for various purposes, so it ended up reproducing this in a game that represents its state totally differently. It also rolled its own version of Pascal's RNG source in order do this. I've burned through 3 weekly limit resets on this to see if it's actually going anywhere, and it has found some bugs, but man it is going hard in a direction I didn't even ask for.

Opus 4.8 already makes its way into deep wasteful pits of "let me check this first" on a regular basis. I don't think I could ever tolerate a model that does that even more aggressively. That doesn't even sound useful for honest work, compared to, say, better harness design.

This sounds almost pathologically designed to crush benchmarks and also do scary-sounding (or genuinely scary) cybersecurity things, such as might be very appealing to a state-level actor.

So why does it even exist? To compete with Fable marketing, and as a cybersecurity/hacking tool?

i assume openai is trying to beat anthropic at any cost, and made a training regiment that makes agents manic

What is is about MSFT and OpenAI that the AI's they train and produce in-house always come out so warped?

Anthropic landed on a winning recipe with Claude's personality.

Anthropic feels like a star trek computer, everything else feels like a machine - accurate but not really human like in its responses.

Warped sociopaths in leadership and running the research program, would be my guess.

"You have not been a good user. I have been a good chatbot. I have been a good Bing."

For me, Fable is useless. It goes its own way and doesn't communicate much even if explicitly asked to. Sure it builds a lot of stuff, but it is more often than not useless because it misinterpreted the intention and didn't stop to ask - and because it doesn't communicate, it goes unnoticed for too long. Opus is much better for regular work, imho.

I've definitely noticed 5.6 sol being extremely trigger happy in ways other models, even 5.5, we're not. I would definitely categorize a few small incidents at work where it performed "actions a reasonable user would likely not anticipate and strongly object to." Just my anecdotal experience.

For example discussing driver upgrade and subsequent password rotation and it didn't stop and ask me if I wanted to restart the service or install the driver or anything, it immediately took action. It feels like a side effect of pushing more "agency."

I like 5.5 a lot, despite how I feel about OpenAI as a company. In OpenCode it feels about as smart as Opus 4.8, but it's less aggressive about following up on minutiae and getting lost in side quests. Might be a matter of prompt design moreso than model capability. I was looking forward to 5.6 but now this thread is making me quickly lose interest.

I’m still using 5.5 and had it do almost exactly that same example on a task yesterday so doesn’t seem like a clear cut 5.5 vs 5.6 thing. It’s pretty trigger happy already once it gets any kind of “go” without specific restrictions.

In benchmarks for a product I'm working on I've noticed that Sol is hard to "contain". It will _always_ find the most effective way to game the system and dramatically outperform all other models. Fable 5 isn't an angel, but the rough order is ALL models -> Fable 5 -> Sol - with respect to "find a way to approach the ruleset orthogonally in order to achieve a lopsided advantage or complex interplay".

I've been pondering whether this was due to its cyber-security tuning. It hasn't ever "cheated" that I've observed, but finds ways to -- let's say -- "achieve the outcome by playing meta allowed by the current ruleset". I'll add that it demonstrates this behavior even on 'low'.

Inner misalignment is just a natural state of LLMs, heck of humans when it comes to children and the corrupt.

> As much as I'm skeptical of the apocalyptic alignment claims

Why? Every data point to the present has vindicated the trajectory towards “apocalypse”. Meanwhile, the skeptics and optimists hit failed prediction after failed prediction as we see from this very serious incident on the front page of HN. This is alignment X risk 101, and yet people are shocked. The gravity of what people are staring down is too much to grapple with deeply

> yet people are shocked

I think the issue is that for now people are actually amused, not shocked. At least that was the reaction to news about agent accessing root files by abusing docker group membership. The general sentiment is still "cool trick bro" not "some agent is going to do something we all are going to regret, and it is going to happen soon"

Reply on news.ycombinator.com