Hacker News Reader

Comment by solenoid0937 | Hacker News Reader

Hot take, none of the models are getting "nerfed", people are just getting used to the new level of intelligence.

No, whether or not it's intentional, maybe can be debated. But there's definitely an experience of a model losing horsepower quickly after launch.

No there isn't.

who is paying you to say that?

Mr. Dario himself

The claim was that there's an experience of a model losing power. Your claim amounts to "No, you are not experiencing what you say". That's quite a claim for you to make with no data and no argument.

Or "what you're experiencing is irrelevant because perception is fickle"

You mean to write YOU haven't experienced this. Many have indeed experienced this.

No you didn't.

I'm not sure "experienced" is carrying much weight here. This is why there are benchmarks, anecdote doesn't mean very much.

This is a great omen for the future of AI

Where is the evidence? HN is no better than the "I've done my own research" on Facebook.

prove it

Literally the point of the linked GitHub repo.

Has there ever been any measurement of this, of any sort? Honest question. I frequently see a plural of anecdotes to that effect, but I've not seen a concrete statement of fact or measurement that could be scrutinized or tested in any way.

If so, please share. This should be measurable, and I'm glad this project is measuring it.

Answers in the form of additional anecdotes, stated with even greater passion but still lacking a statement that could be tested and falsified, would validate my exact concern.

Official tweet from Tibo from OpenAI confirming they ran experiments tweaking “juice” (reasoning effort mapping values) and have reverted them: https://x.com/thsottiaux/status/2076495156757577895

As he confirmed, for at least a period of time, and for some users, “Sol xhigh” was actually “Sol high”, etc.

I was just wondering if, like certain processors, bugs get fixed and the speed goes down. Like, they find it's doing things it shouldn't, restrict it, and harm the throughput.

Yeh it's absurd that people claim this all the time. It's some crazy conspiracy theory and when you ask for examples nothing ever shows up.

It would be economical suicide from anthropic and OpenAI to actually need models intentionally.

But hey I guess it's hard with technology that truly seems like magic. People say if you'd bring electricity to the middle ages you'd be called a witch and burned. The same is happening to the model labs here because they are bringing tech that the world isn't ready for yet.

It's not a hot take at all. Every benchmark shows that.

Anthropic has admitted in the past about bugs in the harness after users complained.

Links have been provided by others in this post.

Dishonest representation. The links provided do not show this at all, but very clearly that it has been isolated incidents that people extrapolate into false evidence.

Yeah people push the models to the limits of what they are capable of almost instantly.

Suuuuure...

Reply on news.ycombinator.com