Hacker News Reader

Comment by steveBK123 | Hacker News Reader

This to me is the big leap from being good at coding to being good at many other tasks.

Coding could be treated as a low stakes (time & money consequences for retries) closed loop system where most other tasks cannot.

If it screws up booking your flight/hotel room, how does the agent verify this, and even if it verifies.. there is an actual cost to changes/cancellations.

Similar with agentic e-commerce, lots of ability to screw that up and just seems ripe for fraud / being picked off by bad actors.

View on HN

Seems like to make agents safe we need tentative, reversible transactions. How do you set up a travel plan and then review it? How do you modify it later?

Unfortunately, travel keeps getting less flexible, with worse cancelation policies.

Because a lot of things in the real world require forward planning, I don't think everything can be just-in-time and tentative/reversible. Some things have to be committed to else you have real consequences and lose your money.

There is no way to do cheap and flexible air travel. If the tickets are cheap, than just a few last minute cancellations can destroy your margin for that flight, if you were to offer them for free. Add in the the risk of your seat reservations being gamed by filling up seats and then canceling, and you get a recipe for disaster.

Trains are usually different because they are much cheaper to operate per trip (for the train operator, not the track operators, but that's a different discussion), so running a half empty train is much less of a problem - especially since you don't need to plan for how much fuel to use ahead of time.

Right, these are actual physical real world problems that aren't going away just because it would be easier for agentic workflows.

Another example - agentic food ordering. How much more convenient would this make your life vs how much of an error rate would you tolerate if the cost/repercussions are on you?

Would a customer be happy if 2% of the time it sends 20 pizzas to a random address in their contacts list instead of 2 pizzas to their own home? Or 5% of the time it completely ignores your dietary restrictions/allergies and orders an entire meal of food you explicitly told it you cannot eat?

Real world problems don't go away just because it would make the tech neater & tidier.

To reply to myself here..

I can STILL replicate this behavior in Google AI summaries 10% of the time:

"is <SOMEPLANT> ok for cats"

to which it replies: "Yes, <SOMEPLANT LONG SCIENTIFIC NAME VERBOSE PHRASING> is toxic for cats"

The other one going around this weekend: "how long hot dogs on grill"

Summary: "The hot dogs on your grill are likely around 5-6 inches long .. "

So scale this category of error to unsupervised agents with access to your credit card.

This repros nearly 100% of the time on most LLMs, even the most advanced ones: https://share.gemini.google/u9NwYu7lbgxe

n=1 but I gave this to Sonnet 5 medium effort (free model) and it had no trouble with it

Try it without "reasoning". As you can see in my example (and GP), it meanders to correctness eventually after emphatically being wrong, and most reasoning modes hide that from you.

If LLMs worked the way people want to believe they do, there’d be no reason to start in the wrong place — a computer should have the facts!

Reply on news.ycombinator.com