Two Concepts of Intelligence

The turkey is fed by the farmer every morning at 9 AM.

Day 1: Fed. (Inductive confidence rises)

Day 100: Fed. (Inductive confidence is near 100%)

Day 250: The farmer comes at 9 AM... and cuts its throat. Happy thanksgiving.

The Turkey was an LLM. It predicted the future based entirely on the distribution of the past. It had no "understanding" of the purpose of the farmer.

This is why Meyer's "American/Inductive" view is dangerous for critical software. An LLM coding agent is the Inductive Turkey example. It writes perfect code for 1000 days because the tasks match the training data. On Day 1001, you ask for something slightly out of distribution, and it confidently deletes your production database because it added a piece of code that cleans your tables.

Humans are inductive machines, for the most part, too. The difference is that, fortunately, fine-tuning them is extremely easy.

I agree with Dijkstra on this one: “The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.”

Here's a paper from September 2025 that compares programs for (a) semantic equivalence (do they do the same thing) and (b) syntactic similarity (are the parse trees similar).

LLMs are more likely to judge programs (correctly or incorrectly) as being semantically equivalent when they are syntactically similar, even though syntactically similar programs can actually do drastically different things. In fact LLMs are generally pretty bad at program equivalence, suggesting they don't really "understand" what programs are doing, even for a fairly mechanical definition of "understand".

https://arxiv.org/pdf/2502.12466

While this is a point in time study and I'm sure all these tools will evolve, this matches my intuition for how LLMs behave and the kinds of mistakes they make.

By comparison the approach in this article seems narrow and doesn't explain a whole lot, and more importantly doesn't give us any hypotheses we can actually test against these systems.

Memory foam doesn't really "remember" the shape of my rear end but we all understand the language games at play when we use that term.

The problem with the AI discourse is that the language games are all mixed up and confused. We're not just talking about capability, we're talking about significance too.

This reads like your standard issue prestige publication essay, as in its an exercise to name drop as many famous people and places the author was involved with.

The whole purpose is not to inform or provoke thought, but for the whole thing to exude prestige and exclusivity, like an article you'd find in a magazine in a high-end private clinics waiting room.

Part of the confusion stems from implicitly pulling in the concept of consciousness into the definition of intelligence.

There is an interiority to our thought life. At least I know there is for myself, because I know what it's like to experience the world as me. I assume that other humans have this same kind of interiority as me, because they are humans like me. And then animals to greater or lesser extent based on how similarly they behave or sense the world around them to humans.

But if there is an "interiority" for LLMs, it must be very very different to humans. The reasoning of an LLM springs into existence for every prompt, then goes away entirely for the next prompt, starting over again from scratch.

Yes this is an over simplification. The LLM has been trained with all kinds of knowledge about the world that persists between invocations. But if the floating point numbers are just sitting there on a disk or other storage medium, it doesn't seem possible that it could be experiencing anything until called into use again.

And the strangeness of the LLM having a completely transformed personality and biases based solely on a few sentences in a prompt. "You are a character in the Lord of the Rings..."

I think this is the sense in which many people argue that an LLM is not "intelligent". It's really an argument that an LLM does not experience the world anything like the way a human being does.

I'd argue you can have much more precise definition than that. My definition of intelligence would be a system that has an internal of a particular domain, and it uses this simulation to guide its actions within that domain. Being able to explain your actions is derived directly from having a model of the environment.

For example, we all have an internal physics model in our heads that's build up through our continuous interaction with our environment. That acts as our shared context. That's why if I tell you to bring me a cup of tea, I have a reasonable expectation that you understand what I requested and can execute this action intelligently. You have a conception of a table, of a cup, of tea, and critically our conception is similar enough that we can both be reasonably sure we understand each other.

Incidentally, when humans end up talking about abstract topics, they often run into exact same problem as LLMs, where the context is missing and we can be talking past each other.

The key problem with LLMs is that they currently lack this reinforcement loop. The system merely strings tokens together in a statistically likely fashion, but it doesn't really have a model of the domain it's working in to anchor them to.

In my opinion, stuff like agentic coding or embodiment with robotics moves us towards genuine intelligence. Here we have AI systems that have to interact with the world, and they get feedback on when they do things wrong, so they can adjust their behavior based on that.

Two concepts of intelligence and neither have remotely anything to do with real intelligence, academics sure like to play with words. I suppose this is how they justify their own existence; in the absence of being intelligent enough to contribute anything of value, they must instead engage in wordplay that obfuscates the meaning of words to the point nobody understands what the hell they're talking about, and confuses the lack of understanding of what they're talking about for the academics being more intelligent than the reader.

Intelligence, in the real world, is the ability to reason about logic. If 1 + 1 is 2, and 1 + 2 is 3, then 1 + 3 must be 4. This is deterministic, and it is why LLMs are not intelligent and can never be intelligent no matter how much better they get at superficially copying the form of output of intelligence. Probabilistic prediction is inherently incompatible with deterministic deduction. We're years into being told AGI is here (for whatever squirmy value of AGI the hype huckster wants to shill), and yet LLMs, as expected, still cannot do basic arithmetic that a child could do without being special-cased to invoke a tool call. How is it that we can go about ignoring reality for so long?

This article highlights how experts disagree on the meaning of (non-human) intelligence, but it dismisses the core problem a bit too quickly imo -

“LLMs only predict what a human would say, rather than predicting the actual consequences of an action or engaging with the real world. This is the core deficiency: intelligence requires not just mimicking patterns, but acting, observing real outcomes, and adjusting behavior based on those outcomes — a cycle Sutton sees as central to reinforcement learning.” [1]

An LLM itself is a form of crystallized intelligence, but it does not learn and adapt without a human driver, and that to me is a key component of intelligent behavior.

[1] https://medium.com/@sulbha.jindal/richard-suttons-challenge-...

Intelligence as described is not the entire "requirement" for intelligence. There are probably more layers here, but I see "intelligence" as the 2nd layer, and beneath that later is comprehension which is the ability to discriminate between similar things, even things trying to decieve you. And at layer zero is the giant mechanism pushing this layered form of intelligence found in living things is the predator / prey dynamic that dictates being alive or food for something remaining alive.

"Intelligence in AI" lacks any existential dynamic, our LLMs are literally linguistic mirrors of human literature and activity tracks. They are not intelligent, but for the most part we can imagine they are, while maintaining sharp critical analysis because they are idiot savants in the truest sense.

Every single word around actual intelligence has been hijacked by the industry. Everything should be prefixed with "Artificial", like vegan food that mimics other food. Or just use more accurate words. Or make them up if they don't exist.

Every thing I used to come to HN for (learning, curiosity, tech, etc) has been mostly replaced by an artificial version of the thing. I still get tricked daily by titles I think are referring a real thing, but turn out to be about AI.

This is kind of bait-and-switch, no?

The author defines American style intelligence as "the ability to adapt to new situations, and learn from experience".

Then argues that the current type of machine-learning driven AI is American style-intelligent because it is inductive, which is not what was supposedly (?) being argued for.

Of course current AI/ML models cannot adapt to new situations and learn from experience, outside the scope of its context window, without a retraining or fine-tuning step.

intelligence/understanding is when one can postulate/predict/calculate/presume something correctly, from concepts about it, without that thing (or similar) ever been in the training/past (or even ever-known).

Yeah, not all humans do it. It's too energy expensive, biological efficiency wins.

As of ML.. Maybe next time, when someone figures out how to combine deductive with inductive, in zillion small steps, with falsifying built-in.. (instead of confronting them 100% one against 100% the other)

Does a French have intelligence? Ah, but we are getting ahead of ourselves. What is a French? Is it a person who speaks French? Is it a person from France? Is your friend René who, tragically, was knocked unconscious on the voyage from Algerie to France and has been in a coma ever since, now twelve years, who eight years ago got French citicenship despite never having been conscious while in Metropolitan France, a French? Is it about ethnicity, culture, borders, phrenology, heritage? You see, I had a French friend ones, a penpal. Oh! How I long for him, Zelda—both my polar opposite and my enigmatic equal. He was equivalent parts analytic, emotional; inquisitive, cooly indifferent; arrogant Anglophone, dainty European; Analytical, Continental; colonial, post-colonial. Every day he challenged me to be better. To be more eclectic, more musing, more deniably off-topic. I would mention irrelevant intellectuals from the 19th century on some tenuous pretext, like they were all confidently chauvinistic in that Edwardian-but-European fashion or some such other thing of that nature; he would mention somewhat obscure intellectuals, at least thrice-removed by association relative to mine, and so irrelevant that it was frankly insulting to your intelligence—but all deniable, all weaved through eight paragraphs of circumlocutions, a challenge for even the most hardened of hermeneutenecists.

And it turned out that he was, as the Slavs say,—and having no better guidance Le Académie Française I am afraid I have to say—a robot. And I loved him just as much as any supposed human I have met.

But does a robot have intelligence? Ah, but we are getting ahead of ourselves.

Marx is fair play, but one of the most prominent cases of understanding everything in advance is undoubtedly Chomsky's theory of innate/universal grammar, which became completely dominant on guess which side of the pond.

I do have to note that the guy writing this is the father of 'modern' OOP industry (the one with endless books about design patterns, UMLs, 'clean code'), something I hope feels like a shameful bit of history of our profession to the current generation of engineers, not something they actively have to engage with.

I don't see this article doing anything to help define intelligence in a useful way.

1) Defining "intelligence" as ability to "understand" isn't actually defining it at all, unless you have a rigorous definition of what it means to understand. It's basically just punting the definition from one loosely defined concept to another.

2) The word "intelligence", in common usage, is only loosely defined, and heavily overloaded, and you'll get 10 different definitions if you ask 10 different people. It's too late to change this, since the meaning of words comes from how they are used. If you want to know the various ways the word is used then look in a dictionary. These are literally the meanings of the word. If you want something more precise then you are not looking for the meaning of the word, but rather trying to redefine it.

3) When we talk about "intelligence" with regards to AI, or AGI, it seems that what people really want to do is to define a new word, something like "hard-intelligence", something rigorously defined, that would let us definitively say whether, or to what degree, an "intelligent" system (animal or machine) has this property or not.

Of course to be useful, this new word "hard-intelligence" needs to be aligned with what people generally mean by "intelligence", and presumably in the future the one of the dictionary senses of "intelligence" will be hard-intelligence.

I think the most useful definition of this new word "hard-intelligence" is going to be a functional one - a capability (not mechanism) of a system, that can be objectively tested for, even with a black box system. However, since the definition should also align with that of "intelligence", which historically refers to an animal/human capability, then it seems useful to also consider where does this animal capability come from, so that our definition can encompass that in most fundamental way possible.

So, with that all said, here's how I would define "hard-intelligence", and why I would define it this way. This post is already getting too long, so I'll keep it brief.

The motivating animal-based consideration for my definition is evolution, and what is the capability that animals evolved to possess intelligence (to varying degrees) have that other animals do not, and what survival benefit does this bring that compensates for the huge cost of large brains in animals with advanced intelligence?

I consider the essence of evolved animal intelligence to be prediction, which means that the animal is not restricted to reacting to the present, but also can plan for the predicted future, which obviously has massive survival benefit - being able to predict where the food and water will be, how the predator is going to behave, etc, etc.

The mechanics of how functional prediction has evolved in different animals varies from something like a fly, whose hard-coded instincts help it avoid predicted swats (that looming visual input predicts I'm about to be swatted by the cow's tail, so I better move), all the way to up to species like ourselves where we can learn predictive signals, outcomes, and adaptive behaviors, rather than these being hard coded. It is widely accepted that our cortex (and equivalent in birds) is basically a prediction machine, which has evolved under selection pressure of developing this super-power of being able to see into the future.

So, my definition of "hard-intelligence" is degree of ability to use, and learn from, past experience to successfully predict the future.

That's it.

There are of course some predictive patterns, and outcomes, that are simple to learn and recognize, and others that are harder, so this is a matter of degree and domain, etc, but at the end of the day it's an objective measure that can be tested for - given the same experiential history to learn from, can different systems correctly predict the continuations of new inputs that follow a similar pattern.

This definition obviously captures the evolutionary super-power of predicting the future, which is at least one of the things that intelligent animals can do, but my assertion, on which the utility of this definition of "hard-intelligence" is based, is that prediction is in fact the underlying mechanism of everything that we consider as "intelligent" behavior. For example, reasoning and planning is nothing more than predicting the outcomes of a sequence of hypothetical what-ifs.

tl/dr - "intelligence" is too fuzzy of a concept to be useful. We need to define a new rigorously defined word to discuss and measure machine (and animal) intelligence. I have suggested a definition.

I had high hopes for this essay because I’ve tried many times to get people online to articulate what they mean by “it doesn’t really understand, it only appears to understand” — my view is that all these arguments against the possibility of LLMs thinking apply equally well to human beings because we don’t understand the process that produces human thinking either.

But the essay is a huge letdown. The European vs. American framing obscures more than it illuminates. The two concepts of intelligence are not really analyzed at all — one could come up with interpretations under which they’re perfectly compatible with each other. The dismissal of Marx and Freud, two of the deepest thinkers in history, is embarrassing, saying a lot more about the author than about those thinkers.

(For anyone who hasn't read much Freud, here's a very short essay that may surprise you with its rigor: https://www.marxists.org/reference/subject/philosophy/works/...)

It's the same kind of schizm that has lead to a lot of hate and mass murder over the last century or so: Abstraction/dimensionality reduction vs. concrete logic and statistics

Concrete statistician: I can learn the problem in its full complexity, unlike the dumdum below me.

Abstract thinker: I understand it, because I can reduce its dimensity to a small number of parameters.

CS: I can predict this because I have statistics about its past behavior.

AT: I told you so.

CS: You couldn't possibly know this, because it has never happened before. You suffer from the hindsight bias.

AT: But I told you.

CS: It has never happened, you couldn't possibly have statistics of when such things occur. You were just lucky.

CS: I'm smart, I can be taught anything

AT: You are stupid because you need to be taught everything.

War (or another sort of mass death or other kind of suffering) emerges.