Hacker News Reader

Comment by timeattack | Hacker News Reader

Thing that I don't understand about LLMs at all, is that how it is possible to for it to "understand" and reply in hex (or any other encoding), if it is a statistical "machine"? Surely, hex-encoded dialogues is not something that is readily present in dataset? I can imagine that hex sequences "translate" to tokens, which are somewhat language-agnostic, but then why quality of replies drastically differ depending on which language you are trying to commuicate with it? How deep that level of indirection goes? What if it would be double-encoded to hex? Triple?

If someone has insight, can you explain please?

View on HN

How I see LLMs (which have roots in early word embeddings like word2vec) is not as statistical machines, but geometric machines. When you train LLMs you are essentially moving concepts around in a very high dimensional space. If we take a concept such as “a barking dog” in English, in this learned geometric space we have the same thing in French, Chinese, hex and Morse code, simply because fundamental constituents of all of those languages are in the training data, and the model has managed to squeeze all their commonalities into same regions. The statistical part really comes from sampling this geometric space.

> not as statistical machines, but geometric machines. When you train LLMs you are essentially moving concepts around in a very high dimensional space.

That's intriguing, and would make a good discussion topic in itself. Although I doubt the "we have the same thing in [various languages]" bit.

What do you mean, exactly, about the doubting part? I thought it was fairly well known that LLMs possess superior translation capabilities.

Sometimes you do not have the same concepts - life experiences are different.

Mother/water/bed/food/etc easily translates into most (all?) languages. Obviously such concepts cross languages.

In this analogy they are objects in high dimensional space, but we can also translate concepts that don’t have a specific word associated with them. People everywhere have a way to refer to “corrupt cop” or “chess opening” and so forth.

> Mother/water/bed/food/etc easily translates into most (all?) languages. Obviously such concepts cross languages.

See also: Swadesh List and its variations (https://en.wikipedia.org/wiki/Swadesh_list), an attempt to make a list of such basic and common concepts.

"Bed" and "food" don't seem to be in those lists though, but "sleep" and "eat" are.

That part I understand and it is quite easy to imagine, but that mental model means that novel data, not present in dataset in a semantical sense, can not be mapped to any exact point in that latent space except to just random one, because quite literally this point does not exist in that space, so no clever statistical sampling would be able to produce it from other points. Surely, we can include hex-encoded knowledge base into dataset, increase dimensionality, then include double-hex encoding and so on, but it would be enough to do (n+1) hex encoding and model would fail. Sorry that I repeat that hex-encoding example, you can substitute it with any other example. However, it seems that our minds do not have any built-in limit on indirection (rather than time & space).

> novel data, not present in dataset in a semantical sense

This is your error, afaik.

The idea of the architecture design / training data is to produce a space that spans the entirety of possible input, regardless of whether it was or wasn't in the training data.

Or to put it another way, it should be possible to infer a lot of things about cats, trained on the entirety of human knowledge, even if you leave out every definition of cats.

See other comments about pre-decoding though, as expect there are some translation-like layers, especially for hardcodable transforms (e.g. common, standard encodings).

People seem to get really hung up on the fact that words have meaning to them, in regards to thinking about what an LLM is doing.

It creates all sorts of illusions about the model having a semantic understanding of the training data or the interaction with the users. It's fascinating really how easily people suspend disbelief just because the model can produce output that is meaningful to them and semantically related to the input.

It's a hard illusion to break. I was discussing usage of LLM by professors with a colleague who teaches at a top European university, and she was jarred by my change in tone when we went from "LLMs are great to shuffle exam content" (because it's such a chore to do it manually to preclude students trading answers with people who have already taken a course) to "LLMs could grade the exam". It took some back and forth for me to convince her that language models have no concept of factuality and that some student complaining about a grade and resulting in "ah ok I've reviewed it and previously I had just used an LLM to grade it" might be career ending.

I think there's a strong case to be made that the detailed map is indeed the land it maps.

Or that one can construct a surprisingly intuitive black box out of a sufficiently large pile of correlations.

Because what is written language, if not an attempt to map ideas we all have in our heads into words? So inversely, should there not be a statistically-relevant echo of those ideas in all our words?

Yeah people keep saying "the map is the territory" when it comes to LLMs presumed ability to reason, but that's nonsense.

> Thing that I don't understand about LLMs at all, is that how it is possible to for it to "understand" and reply in hex (or any other encoding), if it is a statistical "machine"

It develops understanding because that's the best way for it to succeed at what it was trained to do. Yes, it's predicting the next token, but it's using its learned understanding of the world to do it. So this it's not terribly surprising if you acknowledge the possibility of real understanding by the machine.

As an aside, even GPT3 was able to do things like english -> french -> base64. So I'd ask a question, and ask it to translate its answer to french, and then base64 encode that. I figured there's like zero chance that this existed in the training data. I've also base64 encoded a question in spanish and asked it, in the base64 prompt, to respond in base64 encoded french. It's pretty smart and has a reasonable understanding of what it's talking about.

This comment will be very down voted. It is statistically likely to invoke an emotional response.

This depends on how you define the word but I don’t think it’s right to say a “statistical machine” can’t “understand”, after all the human brain is a statistical machine too, I think we just don’t like applying human terms to these things because we want to feel special, of course these don’t work in the same way as a human but they are clearly doing some of the same things that humans do

(this is an opinion about how we use certain words and not an objective fact about how LLMs work)

I agree, I think we keep coming up with new vague things that make us special but it reminds me of the reaction when we found out we were descended from apes.

I don't think we _really_ know whether brain is statistical machine or not, let alone whatever we call by consciousness, so it's a stretch to say that LLMs do some of the things humans do [internally and/or fundamentally]. They surely mimic what humans do, but whether is it internally the same or partly the same process or not remains unknown.

Distinctive part is hidden in the task: you, being presented with, say, triple-encoded hex message, would easily decode it. Apparently, LLM would not. o1-pro, at least, failed spectacularly, on the author's hex-encoded example question, which I passed through `od` twice. After "thinking" for 10 minutes it produced the answer: "42 - That is the hidden text in your hex dump!". You may say that CoT should do the trick, but for whatever reason it's not working.

I was going to say this as well. To say the human brain is a statistical machine is infinitely reductionistic being that we don't really know what the human brain is. We don't truly understand what consciousness is or how/where it exists. So even if we understand 99.99~ percent of the ohaycial brain, not understanding that last tiny fraction of it that is core consciousness means what we think we know about it can be up ended by the last little (arguably the largest though) bit. It's similar to saying you understand the inner working and intricacies of the life and society of new York city because you memorized the phone book.

Not an expert but Sam Harris says consciousness does not exist

Well if Sam Harris says it.

I enjoy eastern philosophy but I'm not a fan of Harris. Why would he charge so much if he truly believes in reducing suffering?

Maybe he wants to reduce his suffering.

This is my point. He said, they said, studies show, but we really have no idea. There's evidence for the fact that Co consciousness isn't even something we posses so much as a universal field we tap into similar to a radio picking up channels. That the super bowl is experienced by your television, but isn't actually contained within it.

To the contrary. Sam Harris often describes consciousness as an indisputable fact we all experience. Perhaps you got it confused with free will.

What I'm trying to say (which deviates from the initial question I've asked), is that biological brains (not just humans, plenty of animals as well) are able to not only use "random things" (whether they are physical or just in mind) as tools, but also use those tools to produce better tools.

Like, say, `vim` is a complex and polished tool. I routinely use it to solve various problems. Even if I would give LLM full keyboard & screen access, would be able to solve those problems for me? I don't think so. There is something missing here. You can say, see, there are various `tools` API-level integrations and such, but is there any real demonstration of "intelligent" use of those tools by AI? No, because it would be the AGI. Look, I'm not saying that AI would never be able to do that or that "we" are somehow special.

You, even if given something as crude as `ed` from '73 and assembler, would be able to write an OS, given time. LLMs can't even figure out `diff` format properly using so much time and energy that none of us would ever have.

You can also say, that brains do some kind of biological level RL driven by utility function `survive_and_reproduce_score(state)`, and it might be true. However given that we as humankind at current stage do not needed to excert great effort to survive and reproduce, at least in Western world, some of us still invent and build new tools. So _something_ is missing here. Question is what.

Same way it understands chinese - except instead of having to both understand the language and a different character set, this is "merely" a transposition cipher.

It is a statistical machine but it is not over bare words/tokens, it effectively operates over a “concept space” that it learns during training.

(Granted the definition of “statistical machine” is quite vague and different folks might define that differently…)

Latent space, no?

There's an encoding, processing, and decoding element to this.

The encoding puts the information into latent vector representations. Then the information is actually processed in this latent space. You are working on highly compressed data. Then there's decoding which brings it back to a representation we understand. This is the same reason you can highly train on one language and be good at translation.

This is over simplified as everything is coupled. But it can be difficult to censor because the fun nature of high dimensional spaces in addition to coupling effects (superposition)

I agree. And i think other comments dont understand how utterly difficult this is. I think that there is a translation tool underneath that translates into English. I wonder if it can also figure out binary ascii or rot13 text. Hex to letter would be a very funky translation tool to have

Try asking them to translate text. You can ask it a question in one language and request the response in another. These are far harder problems than basic encoding, which is just mapping one set of symbols to another.

My Occam's Razor guess: There might be some processing being done before the input is passed to the LLM, and some processing before the response is sent back to the user.

Something like a first pass on the input to detect language or format, and try to do some adjustments based on that. I wouldn't be surprised if there's a hex or base64 detection and decoding pass being done as pre-processing, and maybe this would trigger a similar post-processing step.

And if this is the case, the censorship could be running at a step too late to be useful.

It is responding with a structure of Tokens, and for each node in the structure, it is selecting appropriate tokens according to the context. Here, context means winnie the pooh in hex, so it responds with tokens that resemble that context. The censorship was for a very commonly used context, but not for all contexts.

It is not an statistical machine. I see it repeated constantly. It is not. A statistical machine could be a bayesian spam filter. The many layers and non linear functions between layers create complex functions that go well beyond what you can make with “just” statistics.

Any machine learning model is pretty much just a regression function - its a best guess at what the outputs should be given the inputs.

I’m sure the gazillions of online references to the ASCII Table have something to do with it… no?

Or even conversations presented entirely hex. Not only could that have occurred naturally in the wild (pre-2012 Internet shenanigans could get pretty goofy), it would be an elementary task to represent a portion of the training corpus in various encodings.

So the things I have seen in generative AI art lead me to believe there is more complexity than that. Ask it do a scifi scene inspired by Giger but in the style of Van Gough. Pick 3 concepts and mash them together and see what it does. You get novel results. That is easy to undert5stand because it is visual.

Language is harder to parse in that way. But I have asked for Haiku about cybersecurity, work place health and safety documents in Shakespearean sonnet style etc. Some of the results are amazing.

I think actual real creativity in art, as opposed to incremental change or combinations of existing ideas, is rare. Very rare. Look at style development in the history of art over time. A lot of standing on the shoulders of others. And I think science and reasoning are the same. And that's what we see in the llms, for language use.

There is plenty more complexity, but that emerges more from embedding, where the less superficial elements of information (such as syntactic dependencies) allow the model to hone in on the higher-order logic of language.

e.g. when preparing the corpus, embedding documents and subsequently duplicating some with a vec where the tokens are swapped with their hex repr could allow an LLM to learn "speak hex", as well as intersperse the hex with the other languages it "knows". We would see a bunch of encoded text, but the LLM would be generating based on the syntactic structure of the current context.

In that it created a circuit inside the shoggoth where it translates between hex and letters, sure, but this is not a straight lookup, it's not like a table, any more than that I know "FF " is 255. This is not stochastic pattern matching any more than my ability to look at a raw hex and see structures, ntfs File records and the like (yes, I'm weird, I've spent 15 years in forensics)- in the same way that you might know some French and have a good guess at a sentence if your English and Italian is fluent.

I mean, humans blurt out things they don't mean when they're hurt/angry/stressed, and I know we hate anthropomorphizing LLMs, but that does happen.

sure, i can explain it. it's called "general intelligence"

Reply on news.ycombinator.com