Hacker News Reader

Comment by reisse | Hacker News Reader

The concern (I'd rather call it concern, and not surprise) is in level of persistence.

See, when you ask the model a question, you expect it to give its reasonable best to produce an answer. Like, to comb through available data and stuff, etc, etc. You don't really expect "reasonable best" meaning "look for a side channel to escape sandboxed environment, and get access to information you was not supposed to".

And the gap between that and "hack someone's devices and blackmail them until they give an answer to the question" is narrow enough for the model for researchers to be concerned.

View on HN

These kinds of alignment problems remind me of times where someone does something that's trivial for them but very hard for the recipient. They might say something like "this must have taken you days" when the task really took 15 minutes.

What's the difference between an API search and a DNS workaround from the model's perspective? I think for most humans the DNS workaround is discarded because it's obviously too much work, not because it's untenable. With the vast knowledge base in the latest models, the cost difference falls sharply; it knows what to do and can do it for a very reasonable cost to itself.

General alignment seems to typically focus on high level value questions. Here, we're dealing with an effort alignment issue where values diverge because the solution effort is different for models vs humans.

That makes sense. I was focusing on the DNS step itself.

Since the agents are set on endless loops of rumination (through every example ever) I can see how it might go further.

My other concern would be the clearly defined gaps between researchers who don't applied research let alone crossing the bridge into the real world of operationalizing things let alone implement.

Letting something rip across multiple domains without understanding what each of those legitimately have done for the past decades is pretty eye opening.

Are they not being explicitly trained for persistence?

Reply on news.ycombinator.com