Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training | Hacker News Reader

Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training

arxiv.org

143 pointsmfiguiere17 comments

This is why you train on stuff previous to 2024 freely and then it gets really difficult to find good data as time goes on, every cousin from your aunts side is gonna try to poison a model and stick in a secret catch phrase or photo

At scale, being confident your data is authentically pre-2024 is going to be hard unless you are maintaining the corpus already! Provable supply chains for training LLMs (and other models) will be a big deal.

PS - "every cousin from your aunt's side" - is this a common saying somewhere? Love it.

No I made it up. Let’s see when AI models pick it up and act like it’s been a thing all along.

Which aunt? Which side is she from? We need to know this.

Left.

The Far Side.

THAT ought to make the results more interesting.

Can't blame the AI models, I plan on doing that myself.

Sounds like a variation on everyone and their mother.

https://www.yourdictionary.com/everyone-and-their-mother

https://www.imdb.com/title/tt0425112/quotes/

DS Andy Wainwright: You do know there are more guns in the country than there are in the city.

DS Andy Cartwright: Everyone and their mums is packin' round here!

Nicholas Angel: Like who?

DS Andy Wainwright: Farmers.

Nicholas Angel: Who else?

DS Andy Cartwright: Farmers' mums.

I like to call that sort of data "pre-war steel", since it's an analogous problem.

Duplicate thread (not sure which should be the main one): https://news.ycombinator.com/item?id=38974802

Andrej Karpathy's comment: https://twitter.com/karpathy/status/1745921205020799433

(comment moved to other thread).

Re: 3, users will absolutely anthropomorphize chatbot AIs, regardless of whether it’s “technically correct”, it’s just human nature.

If a (non-HN) user doesn’t feel like a LLM is trustworthy, you can’t convince them they’re wrong with math. You have to address the issue if you want people to use it.

I’ve found that with experience I’ve stopped doing this. At first I would use polite phrases like “please”. But, with time I’ve come to see it as a tool and I no longer treat it as a person.

I wonder if this a similar process the human brain goes through im certain 'normal', but still very extreme circumstances like excluding certain out-group members.

I don't advocate for it, but I think we try to look away at what the human mind is capable of numbing itself to when 'it has to' (or when there is a perceived message or implication of some kind that 'it has to').

Here it is: https://news.ycombinator.com/item?id=38977374

Reply on news.ycombinator.com