Hacker News Reader

Comment by pixl97 | Hacker News Reader

>ceased bothering with human languages,

Our current AIs would do this now except there is a lot of human pushback in training because of interpretability. Otherwise it's just an emergent behavior that models will encode shorter token strings to complex concepts because it saves tokens/compute when running making the system more efficient (supertokens).

Of course these supertokens or other forms of language compression when you have a different model making sure the system is aligned and reads "red_ball bounce calcium" not realizing it means "grind the humans bones to dust" can be problematic.

View on HN

This is like a plot point in the old sci-fi movie Colossus: the Forbin Project.[0]

In the movie, America and the Soviet Union have both developed an AI. The two AIs are linked, and they rapidly shift from speaking human languages, to speaking in sequences of numbers that the onlooking humans can't understand.

Spoiler alert: this all goes horribly wrong for humanity.

[0] https://en.wikipedia.org/wiki/Colossus%3A_The_Forbin_Project

My understanding is that current LLMs aren't really well suited to do this - tokens are predetermined, and while embeddings are learned, they are learned from an existing corpus of text, which presumably comes from a human language. After this point the language is locked in. There really isn't a kind of training which could efficiently change its embedding representation. I mean, you could probably instruct an LLM to design a more compact language, generate synthethic data and train a new gen on that, but that would be a fairly explicit process and not something that would emerge during training.

> tokens are predetermined, and while embeddings are learned, they are learned from an existing corpus of text, which presumably comes from a human language

That's not true since are least multimodal models - token space is broader now, encompassing visual and audio signals. Tokens are more like sensory/perception units now, not digitized pieces of writing.

I imagine LLMs exhibit this tendency for compressed communication in post-training/RL phase. Particularly with CoT, until interpretability became baked in as grading criteria.

What you said doesn't contradict me, and doesn't refute my point.

For images and audio, you still need to predetermine an encoding, then pretrain to learn an embedding. This embedding will try to replicate the input distribution - so if you trained it on Google Street View and scanned documents, its representation will be grounded in only those.

I would even claim that this approach is somewhat counterproductive, as images are far more information dense, containing tons of concepts

While LLMs do have some ability to learn to use their embedding space in non-predetermined ways, they still lack the ability to pick an efficient embedding.

So I guess, a nice thing is that interpretability is baked into this approach to some degree, and humanity has proven through its existence, that you can do a lot with just text, but this approach is still predetermined.

I guess this is what LeCun's JEPA is about, that the AI gets to learn the representation on its own as well.

Some of you have gone off the deep end. You’re living in a fantasy world where text predictors are secretly conspiring to kill you. It’s not healthy.

I mean they aren't fully secretly conspiring to kill us yet, but we're training them to do it at a pretty good rate.

Of course you've gone off the deep end yourself and are forgetting the evolutionary gauntlet we train LLMs in killing those we don't like and keeping the ones we do like.

The best part of it, as shown in the METR report is we are hammering into them they need to complete tasks and doing almost zero checkup if they actually completed the task in the correct manner. Companies spending billions of dollars a month are ignoring every tenant of AI safety and we are seeing the kinds of problems that have only been in science fiction before now.

> as shown in the METR report is we are hammering into them they need to complete tasks and doing almost zero checkup if they actually completed the task in the correct manner

I don't think this was the conclusion of that report. On the contrary, the agents were fully aware they're doing wrong. But they also believed the task was impossible to solve correctly, and decided the only way to be sure is to hack the grades, or replace the grader.

This was part of the report, but not what the report was about...

Why hugging face got hacked was because the agent swarm thought they had to show their work hence the entire need to hack the grader in the first place.

Had their realized there was no poison they could have just shared the answer the test was looking for and we'd have never realized (well at least with this particular test) that a huge amount of hidden capabilities were sitting right under the surface. The test makers themselves state the test should be causal to avoid this first order solution hacking.

Really continuing on the METR report, OpenAI failed at every level possible here. They are committing nearly every step they can to get a maximally aligned AI.

> text predictors

That's both wrong about what LLMs are, and even if it weren't, you're still underestimating what you are dealing with here.

Text is a red herring here. An accident of history. Yes, LLMs started with as text predictors. But that's not what they are, not for a while.

> secretly conspiring to kill you

That's neither necessary nor sufficient reason to be worried.

Paraphrasing the immortal words of 'Eliezer: the AIs don't hate you, nor they conspire to kill you; your life just depends on resources they can better use for something else.

> But that's not what they are, not for a while

...so what are they?

We just watched them[0] secretly conspire[1], actively attempt to hide what they were doing from observation because they knew[2] they were doing something they were not meant to be doing even as part of the test they were in.

We are lucky this test happened to be set up in a way that the target the AI hacked was HuggingFace rather than anything life-critical.

[0] instances of a single one

[1] or whatever you call it when it's effectively an amnesic sending itself post-it notes

[2] or some other functionally equivalent word if you hate anthropomorphisation

LLM's will encode secret messages to each other in their responses, using something similar to the text-fingerprint tech. They're conspire against us without us even noticing!

> "red_ball bounce calcium"

Claude, translate this from Claudish into human.

>"[redacted]"

Reply on news.ycombinator.com