Hacker News Reader

Nah, I think it will be the other way around. We currently have intelligent agents working on help desk and other customer service roles. Those agents have had their acceptable output more and more restricted.

We will just do to LLMs what we are already doing to people.

View on HN

The options are a support LLM that can sometimes be tricked into giving out refunds for items that were never purchased, and a support LLM that never gives out refunds at all. (It might hallucinate that it gave a refund, but it won't be hooked up to any API that actually allows it to do so.)

This is actually the only possible answer IMHO. Humans are Turing-complete, which means the best we can do is give them training and guidelines and trust them. Even so their training can be subverted through social engineering.

What we're talking about here is social engineering of LLMs. That's currently pretty easy. It will get harder but it cannot be made impossible.

You just have to make it hard enough so that the hassle required ain't worth it.

Reply on news.ycombinator.com