Hacker News Reader

Comment by j45 | Hacker News Reader

So the LLM was able to look up a basic way to reroute things to get to their destination (likely well available and trained in the corpus) and it's surprising?

What's surprising is the surprise the security testers are explaining.

By setting an outcome to reach an endpoint, and to find all possible ways there, would this not be in the realm of possibility if an agent is reasonably in control of a vps?

Having the vps locked within a network layer it can't see or get out of is pretty common practice when setting up IaaS / PaaS.. sans-llm.

Maybe I'm missing something here, what confuses me is how something so relatively simple can get such prominent coverage, it's hard to imagine this kind of ability is still relatively new or surprising to folks working at the major models, unless they aren't hiring for network experience?

View on HN

The concern (I'd rather call it concern, and not surprise) is in level of persistence.

See, when you ask the model a question, you expect it to give its reasonable best to produce an answer. Like, to comb through available data and stuff, etc, etc. You don't really expect "reasonable best" meaning "look for a side channel to escape sandboxed environment, and get access to information you was not supposed to".

And the gap between that and "hack someone's devices and blackmail them until they give an answer to the question" is narrow enough for the model for researchers to be concerned.

That makes sense. I was focusing on the DNS step itself.

Since the agents are set on endless loops of rumination (through every example ever) I can see how it might go further.

My other concern would be the clearly defined gaps between researchers who don't applied research let alone crossing the bridge into the real world of operationalizing things let alone implement.

Letting something rip across multiple domains without understanding what each of those legitimately have done for the past decades is pretty eye opening.

Are they not being explicitly trained for persistence?

These kinds of alignment problems remind me of times where someone does something that's trivial for them but very hard for the recipient. They might say something like "this must have taken you days" when the task really took 15 minutes.

What's the difference between an API search and a DNS workaround from the model's perspective? I think for most humans the DNS workaround is discarded because it's obviously too much work, not because it's untenable. With the vast knowledge base in the latest models, the cost difference falls sharply; it knows what to do and can do it for a very reasonable cost to itself.

General alignment seems to typically focus on high level value questions. Here, we're dealing with an effort alignment issue where values diverge because the solution effort is different for models vs humans.

Reply on news.ycombinator.com