Hacker News Reader

Comment by walrus01 | Hacker News Reader

I'm not anthropomorphizing anything, I literally said that the training data for the formulas and equations is baked into it. It only "knows" things because a crawler and scraper acquired the information from an existing written source. In just about the same way that information is baked into a printed encyclopedia.

View on HN

This is not even remotely accurate. "Baking information" like into a "printed encyclopedia" is memorization. It has been shown, time and time again, that LLMs do not merely memorize. It is not even possible for it to do so at scale. It can memorize some things, yes, but it is forced during the training procedure to bake general concepts into intermediate layers (this is why transfer learning works), analogous to compression. One can make several arguments that compression and intrinisic feature sparsity is the closest mathematical explanation to understanding that we have.

It is completely possible to ask an LLM a series of increasingly more esoteric and discrete questions until you find precisely what information did, or did not make it into the model. If you know something rare and the LLM does not, you'll immediately see when it's hallucinating an answer or answering factually.

>>> I'm not anthropomorphizing anything ...

Yes you are, regarding LLMs at least. Here's why:

  just for fun I asked a reasonably smart LLM to ...
  [be] capable of understanding if it's gone off on
  a hallucinatory path ...

"Smart" in this context is a subjective value judgement. "Hallucinations" are only experienced by living organisms.

You then went on to state:

> If you know something rare and the LLM does not, you'll immediately see when it's hallucinating an answer or answering factually.

Again, "hallucinating" is not something an algorithm can do. Also, determining factuality is again subjective based on the person assessing the information.

You ever heard of something called a metaphor, guy? I use the word "smart" as shorthand to describe something that scores highly in a number of coding and terminal use benchmarks (as compared to, let's say, a 30B size model from one and a half years ago.

https://artificialanalysis.ai/leaderboards/models

You will note that in my original comment I very specifically said "deepseek v4 flash 0731", which by many benchmarks/metrics, is "smarter" (again, this is a metaphor) than a smaller or older model. It is also specifically known to be relatively capable of producing code and formulas on demand in several common languages.

And "hallucinating" to mean "outputs plausible sounding gibberish that doesn't hold together consistently". Of course there's no actual hallucination going on.

> You ever heard of something called a metaphor, guy?

In this media (comments in HN threads), all I can do is interpret what people write. ;-)

> And "hallucinating" to mean "outputs plausible sounding gibberish that doesn't hold together consistently". Of course there's no actual hallucination going on.

This may very well be what you know to be true and I have no reason nor desire to assume otherwise. The problem is... Many people use the word "hallucinating" in this context literally and not metaphorically.

Since I do not know you, how am I to tell the difference?

You have 'logic' in your name. You would be well aware of the physical Church Turing statement and as of yet it has held up. Everything, including our brain, as we currently know, is an algorithm.

To be fair, they also have “adieu” (farewell) to logic.

> To be fair, they also have “adieu” (farewell) to logic.

It is always a joy when a person, such as yourself, finds the irony in my moniker.

Thank you for this.

Now this is true, I do agree with this. There is indeed a good amount of memorization that is still taking place; see [1]. But it definitely isn't all memorization, or indeed, majority memorization. And even if we are able to extract things verbatim like this, we do not know how this is stored internally, as this may simply be the text that, with the rest of the internet in context, can be very radically compressed.

But in general, yes, the LLM cannot know about concepts that are far outside of its training set. Humans are the same, I would argue. If you add a good amount of your own knowledge into its context, or better yet, into finetuning, you might find it surprisingly easy to get it caught up on that material.

[1] Ahmed, A., Cooper, A. F., Koyejo, S., & Liang, P. (2026). Extracting books from production language models. arXiv preprint arXiv:2601.02671. https://arxiv.org/abs/2601.02671.

My favorite test is to just ask it to add two very large numbers or do other math of that sort. (this is also part of my favorite answer to the chinese room).

You'd be surprised how few digits you need to make a problem that is presumably unique in earth history. For a typical sum, the number of pre-existing answers would need to scale with 10^n lines of text where n is the number of digits. This expands out of control REALLY quickly. A quick guesstimate has you somehow reading out of a literal black hole at n=21 digits if your LUT is on paper, or n=26 digits if you're using modern HDD technology. O:-)

No, most of a modern LLM's training time is spent in RLVR, which does not "acquire information from an existing source". You can RL behaviors into a randomly initialized neural network.

This is true, but you're not going to get anywhere. The pretraining phase is necessary to immensely reduce variance in the RLVR stage. Once there, RLVR has a surprising tendency to only restrict the generated space further. This is not true of RLHF, by the way, which I find to be particularly fascinating, but I digress.

Reply on news.ycombinator.com