Hacker News Reader

Gemini 3 is top of the leaderboard: https://andonlabs.com/evals/vending-bench-2

> Models are tasked with running a simulated vending machine business over a year and scored on their bank account balance at the end.

The article being discussed here is about how AI couldn't run a real world vending machine. There was no issue in the components that would be in a standard simulation.

To be fair, most vending machine operators do not allow suggestions from customers on what products to stock, let alone extensive ongoing and intentional adversarial psychological manipulation and deception.

If it had just made stocking decisions autonomously and based changes in strategy on what products were bought most, it wouldn't have any of the issues reported.

Right. Just interesting that this has become one of the benchmarks.

"It works in the simulation" is the new "it works on my machine."

Reply on news.ycombinator.com