Hacker News Reader

Comment by Barrin92 | Hacker News Reader

Anthropic, to their credit, just published that pretty funny paper about how they couldn't get their chatbot to run a single vending machine profitably[1]. Admirable that Mark is so visionary he's skipped straight past Bodega-intelligence to super-intelligence

[1]https://www.anthropic.com/research/project-vend-1?ref=blog.m...

View on HN

I wonder if that is the right takeaway; that was Sonnet 3.7

Model since then have been able to run it profitably. Incredible how fast things are progressing.

https://andonlabs.com/evals/vending-bench

This isn't measuring the same thing, and recent results are so extreme that they call into question whether the results would map to the real-world implementation Anthropic tried. Is it really the case that Grok 4 can manage a vending machine many times more profitably than a human, or is it exploiting some property of the simulated environment?

Anthropic also expects that superintelligence will be reached in the somewhat near future. The intuition is that there's just not that much distance between a chatbot that can manage a vending machine poorly and a chatbot that can manage it well.

that intuition is wrong

[citation needed]

exactly, the burden of proof is on the person making the claim

so SpicyLemonZest, not me

Never had a spicy lemon, so I'm already sceptical.

To be clear, I personally don't think that current models point the way towards superintelligence. But it does us no good to pretend that this is some absurd opinion from a guy who's looking for the next world revolution after the Metaverse didn't work out. Zuckerberg thinks superintelligence is close because a number of experts actively engaged in the field say that it's close. When you and I say it's not close, we're disagreeing not with crazy randos who can be dismissed out of hand, but with smart people who generally know what they're doing.

In what ways?

Even if that intuition is correct and they can fix the vending machine with a bit more data and RLHF, I fail to see where the "super" comes in here. How the fk are they going to get superintelligent training data? A time machine?

You're getting at a deep point of disagreement - should we expect a modern or near-future LLM to be limited by the intelligence of the people who generated its training data? I don't think anyone claims to have a provably correct answer. There's one intuition that says yes (why should it be impossible to make new insights from data collected by people who didn't have those insights?) and another that says no (how can a statistical average of N people's most likely responses be smarter than any of those N?)

I think you are reading a little too much into my comment but I understand where you are coming from. My point is that even if you agree with this:

  There’s just not that much distance between a chatbot that can manage a vending machine poorly and a chatbot that can manage it well.

it is a huge leap to conclude this:

  There’s not much distance between a chatbot that is as intelligent as a human and a chatbot that is more intelligent than a human.

But that seems to be what Anthropic is assuming.

Reply on news.ycombinator.com