My personal assessment is that LLMs can do neither.
An LLM has: words in its input plane, words in its output plane, and A LOT of cross-linked internals between the two.
Those internals aren't "words" at all - and it's where most of the "action" happens. It's how LLMs can do things like translate from language to language, or recall knowledge they only encountered in English in the training data while speaking German.
The heavy lifting here is done by embeddings. This does not require a world model or “thought”.
This is a case where it's going to be next to impossible to provide proof that no counterexamples exist. Conversely, if what I've written there is wrong then a single counterexample will likely suffice to blow the entire thing out of the water.
If you're interested in why compression is like understanding in many ways, I'd suggest reading through the wikipedia article on Kolmogorov complexity.
My "abstract thoughts" are a stream of words too, they just don't get sounded out.
Tbf I'd rather they weren't there in the first place.
But bodies which refuse to harbor an "interiority" are fast-tracked to destruction because they can't suf^W^W^W be productive.
Funny movie scene from somewhere. The sergeant is drilling the troops: "You, private! What do you live for!", and expects an answer along the lines of dying for one's nation or some shit. Instead, the soldier replies: "Well, to see what happens next!"
To me, solving problems happens in a logico/aesthetical space which may be the same as when you are intellectually affected by a work of art. I don't remember myself being able to translate directly into words what I feel for a great movie or piece of music, even if in the late I can translate this "complex mental entity" into words, exactly like I can tell to someone how we need to change the architecture of a program in order to solve something after having looked up and right for a few seconds.
It seems to me that we have an inner system that is much faster than language, that creates entities that can then beslowly and sometimes painfully translated to language.
I do note that I'm not sure about any of the previous statements though'
The twist about words in particular is they are distinctly articulable symbols, i.e. you can sound 'em out - and thus, presumably, have a reasonable expectation for bearers of the same language to comprehend if not what you meant then at least some vaguely predictable meaning-cloud associated with the given speech act.
That's unlike e.g. the numbers (which are more compressed, and thus easier to get wrong), or the syntagms of a programming language (which don't even have a canonical sonic representation).
Therefore, it's usually words that are taught to a mind during the formative stages of its emergence. That is, the words that you are taught, your means of inner reflection, are still sort of an imposition from the outside.
Just consider what you life trajectory would've been if in your childhood you had refused to learn any words, or learned them and then refused to mistake them for the things they represent!
Infants and even some animals recognize their reflection in a mirror; however, practically speaking, introspection is something that one needs to be taught: after recognizing your reflection you still need to be instructed what is to be done about it.
Unfortunately, introspection needing to be taught means that introspection can be taught wrongly.
As you can see with the archetypical case of "old and wise person does something completely stupid in response to communication via digital device", a common failure mode of how people are taught introspection (and, I figure, an intentional one!) is not being able to tell apart yourself from your self, i.e. not having an intuitive sense of where the boundary lies between perception and cognition, i.e. going through life without ever learning the difference between the "you" and the "words about you".
It's extremely common, and IMO an extremely factory-farming kind of tragic.
I say it must be extremely intentional as well, because the well-known practice of using "introspection modulators" to establish some sort of perceptual point of reference (such as where the interior logicoaeshtetical space ends and exterior causalityspace begins) very often ends up with the user in, well, a cage of some sort.
> It's extremely common
I cannot conceive this ? I am lacking the empirical knowledge you seem to have. (I don't understand your "archetypical case", I can't relate to it). I'd love a reexplanation of your point here, as your intent is unclear to me.
I didn't understand also the "introspection modulators" part :(, (a well known practice ?? I must be living on another planet haha...).
edit: or maybe that's a metaphor for "language" ??
Hmm, seems unlikely. They are not sounded out part is true, sure, but I question whether 'abstract thoughts' can be so easily dismissed as mere words.
edit: come to think of it and I am asking this for a reason: do you hear your abstract thoughts?
Play a little game of "what word will I think of next?" ... just let it happen. Those word choices are fed to the monologue, they aren't a product of it.
move.panic.fear.run
that effectively becomes one thought and not a word exactly. I am stating it like this, because I worry that my initial point may have been lost.
edit: I can only really speak for myself, but I am curious how people might respond to the distinction.
Most of the fucking time, and I would prefer that I didn't. I even wrote that, lol.
I don't think they're really "mine", either. It's just all the stuff I heard somewhere, coalescing into potential verbalizations in response to perceiving my surroundings or introspecting my memory.
If you are a materialist positivist, well sure, the process underlying all that is some bunch of neural activation patterns or whatever; the words remain the qualia in which that process is available to my perception.
It's all cuz I grew up in a cargo cult - where not presenting the correct passwords would result in denial of sustenance, shelter, and eventually bodily integrity. While presenting the correct passwords had sufficient intimidation value to advance one's movement towards the "mock airbase" (i.e. the feeder and/or pleasure center activation button as provided during the given timeframe).
Furthermore - regardless whether I've been historically afforded any sort of choice in how to conceptualize my own thought processes, or indeed whether to have those in the first place - any entity which has actual power to determine my state of existence (think institutions, businesses, gangs, particularly capable individuals - all sorts of autonomous corpora) has no choice but to interpret me as either a sequence of words, a sequence of numbers, or some other symbol sequence (e.g. the ones printed on my identity documents, the ones recorded in my bank's database, or the metadata gathered from my online represence).
My first-person perspective, being constitutionally inaccessible to such entities, does not have practical significance to them, and is thus elided from the process of "self-determination". As far as anyone's concerned, "I" am a particular sequence of that anyone's preferred representational symbols. For example if you relate to me on the personal level, I will probably be a sequence of your emotions. Either way, what I may hypothetically be to myself is practically immaterial and therefore not a valid object of communication.
Though I do think in human brains it's also an interplay where what we write/say also loops back into the thinking as well. Which is something which is efficient for LLMs.
But raising kids, I can clearly see that intelligence isn't just solved by LLMs
Funny, I have the opposite experience. Like early LLMs kids tend to give specific answers to the questions they don't understand or don't really know or remember the answer to. Kids also loop (give the same reply repeatedly to different prompts), enter highly emotional states where their output is garbled (everyone loves that one), etc. And it seems impossible to correct these until they just get smarter as their brain grows.
What's even more funny is that adults tend to do all these things as well, just less often.
As the person you initially responded to said, observing children growing up should make it obvious.
Or if we shift to stating the obvious, there's the minor detail that the vast majority of architectures lack the ability to learn during inference. That's one of the basic things that biological systems are capable of.
If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.
They’re certainly special both within the individual but also as a species on this planet. There are many similar to human brains but none we know of with similar capabilities.
They’re also most obviously certainly different to LLMs both in how they work foundationally and in capability.
I definitely agree with the materialist view that we will ultimately be able to emulate the brain using computation but we’re nowhere near that yet nor should we undersell the complexity involved.
If I throw some braincells into a cup alongside the dice, will they think about the outcome anymore than the dice alone?
If so, yes, they're thinking
That's not what we have here.
"It is only a fallacy if you "P, therefore C" which GP is not (at least to my eye) doing."
Some people are willfully blind.
"A.I. and humans are as different as chalk and cheese."
As aphorisms are a good way to think about this topic?
From this point on its all about efficiencies:
modeling efficiency: how do we best fit the elephant, with bezier curves, rational polynomials, ...?
memory bandwidth training efficiency: when building coincidence statistics, say bigrams, is it really necessary to update the weights for all concepts? a co-occurence of 2 concepts should just increase the predicted probability for the just observed bigram and then decrease a global coefficient used to scale the predicted probabilities. I.e. observing a baobab tree + an elephant in the same image/sentence/... should not change the relative probabilities of observing french fries + milkshake versus bicycle + windmill. This indicates different architectures should be possible with much lower training costs, by only updating weights of the concepts observed in the last bigram.
and so on with all other kinds of efficiencies.
> Human brains aren't magic, special or different.
[1] https://www.nature.com/articles/s41598-024-62539-5
As the result, all living cells with DNA emit coherent (as in lasers) light [2]. There is a theory that this light also facilitates intercellular communication.
[2] https://www.sciencealert.com/we-emit-a-visible-light-that-va...
Chemical structures in dendrites, not even neurons, are capable to compute XOR [3] which require multilevel artificial neural network with at least 9 parameters. Some neurons in brain have hundredths of thousands of dendrites, we are now talking of millions of parameters only in single neuron's dendrites functionality.
[3] https://www.science.org/doi/10.1126/science.aax6239
So, while human brains aren't magic, special or different, they are just extremely complex.
Imagine building a computer with 85 billions of superconducting quantum computers, optically and electrically connected, each capable of performing computations of a non-negligibly complex artificial neural network.
> We know this because we can test what aspects of neurons actually lead to practical real world effects.
What is wonderful here is that contemporary electronics wizardry that allowed us to have machines that mimic some of thinking, also is very concerned of the quantum-level electromagnetic effects at the transistor level.
How complex our everything computing-related should be to mimic thinking (of humans) little more closely?
Planes and boats disrupt the environments they move through and air and sea freight are massive contributors to pollution.
While I agree to some extent with the materialistic conception, the brain is not an isolated mechanism, but rather the element of a system which itself isn't isolated from the experience of being a body in a world interacting with different systems to form super systems.
The brain must be a very efficient mechanism, because it doesn't need to ingest the whole textual production of the human world in order to know how to write masterpieces (music, litterature, films, software, theorems etc...). Instead the brain learns to be this very efficient mechanism with (as a starting process) feeling its own body sh*t on itself during a long part of its childhood.
I can teach someone to become really good at producing fine and efficient software, but on the contrary I can only observe everyday that my LLM of choice keeps being stupid even when I explain it how it fails. ("You're perfectly right !").
It is true that there's nothing magical about the brain, but I am pretty sure it must be stronger tech than a probabilistic/statistical next word guesser (otherwise there would be much more consensus about the usability of LLMs I think).
Animals and computers come close in some ways but aren't quite there.
“Internal combustion engines and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?”
The question isn't about what an hypothetical mechanism can do or not, it's about whether the concrete mechanism we built does or not. And this one doesn't.
I will absolutely say that all ML methods known are literally too stupid to live, as in no living thing can get away with making so many mistakes before it's learned anything, but that's the rate of change of performance with respect to examples rather than what it learns by the time training is finished.
What is "abstract thought"? Is that even the same between any two humans who use that word to describe their own inner processes? Because "imagination"/"visualise" certainly isn't.
If you consider that LLMs have already "learned" more than any one human in this world is able to learn, and still make those mistakes, that suggests there may be something wrong with this approach...
To a limited degree, they can compensate for being such slow learners (by example) due to the transistors doing this learning being faster (by the wall clock) than biological synapses to the same degree to which you walk faster than continental drift. (Not a metaphor, it really is that scale difference).
However, this doesn't work on all domains. When there's not enough training data, when self-play isn't enough… well, this is why we don't have level-5 self-driving cars, just a whole bunch of anecdotes about various different self-driving cars that work for some people and don't work for other people: it didn't generalise, the edge cases are too many and it's too slow to learn from them.
So, are LLMs bad at… I dunno, making sure that all the references they use genuinely support the conclusions they make before declaring their task is complete, I think that's still a current failure mode… specifically because they're fundamentally different to us*, or because they are really slow learners?
* They *definitely are* fundamentally different to us, but is this causally why they make this kind of error?
Some machines, maybe. But attention-based LLMs aren't these machines.
The same way a todler creeping is the start of the general concept of space exploration.
But to your point, I do see a lot of people very emotionally and psychologically committed to pointing out how deeply magical humans are, and how impossible we are to replicate in silicon. We have a religion about ourselves; we truly do have main character syndrome. It's why we mistakenly thought the earth was at the center of the universe for eons. But even with that disproved, our self-importance remains boundless.
This a straw man, the question isn't if this is possible or not (this is an open question), it's about whether or not we are already here, and the answer is pretty straightforward: no we aren't. (And the current technology isn't going to bring us anywhere near that)
It's not just that. The problem of “deep learning” is that we use the word “learning” for something that really has no similarity with actual learning: it's not just that it converges way too slowly, it's also that it just seeks to minimize the predicted loss for every samples during training, but that's no how humans learn. If you feed it enough flat-earther content, as well a physics books, an LLM will happily tells you that the earth is flat, and explain you with lots of physics why it cannot be flat. It simply learned both “facts” during training and then spit it out during inference.
A human will learn one or the other first, and once the initial learning is made, it will disregards all the evidence of the contrary, until maybe at some point it doesn't and switches side entirely.
LLMs don't have an inner representation of the world and as such they don't have an opinion about the world.
The humans can't see the reality for itself, but they at least know it exists and they are constantly struggling to understand it. The LLM, by nature, is indifferent to the world.
This is a terrible example, because it's what humans do as well. See religious, or indeed military, indoctrination. All propaganda is as effective as it is, because the same message keeps getting hammered in.
And not just that, common misconceptions abound everywhere and not just conspiracy theories, religion, and politics. My dad absolutely insisted that the water draining in toilets or sinks are meaningfully influenced by the Coriolis effect, used an example of one time he went to the equator and saw a demonstration of this on both sides of the equator. University education and lifetime career in STEM, should have been able to figure out from first principles why the Coriolis effect is exactly zero on the equator itself, didn't.
> A human will learn one or the other first, and once the initial learning is made, it will disregards all the evidence of the contrary, until maybe at some point it doesn't and switches side entirely.
We don't have any way to know what a human would do if they could read the entire internet, because we don't live long enough to try.
The only bet I'd make is that we'd be more competent than any AI doing the same, because we learn faster from fewer examples, but that's about it.
> LLMs don't have an inner representation of the world and as such they don't have an opinion about the world.
There is evidence that they do have some inner representation of the world, e.g.:
You completely misread my point.
The key thing with humans isn't that they cannot believe in bullshit. They can definitely do. But we don't usually believe in both the bullshit and in the fact the BS is actually BS. We have opinions on the BS. And we, as a species, routinely die or kill for these opinions, by the way. LLM don't care about anything.
I can't parse what you mean by this.
> LLM don't care about anything.
"Care" is ill-defined. LLMs are functions that have local optima (the outputs); those functions are trained to approximate other functions (e.g. RLHF) that optimise other things that can be described with functions (what humans care about). It's a game of telephone, like how Leonard Nimoy was approximating what the script writers were imagining Spock to be like when given the goal of "logical and unemotional alien" (ditto Brent Spiner, Data, "logical and unemotional android"), and yet humans are bad at writing such characters: https://tvtropes.org/pmwiki/pmwiki.php/Main/StrawVulcan
But rather more importantly in this discussion, I don't know what you care about when you're criticising AI for not caring, especially in this context. How, *mechanistically*, does "caring" matter to "learning abstract thought", and the question of how closely LLMs do or don't manage it relative to humans?
I mean, in a sense, I could see why someone might argue the exact opposite, that LLMs (as opposed to VLMs or anything embodied in a robot, or even pure-text agents trained on how tools act in response to the tokens emitted) *only* have abstract "thought", in so far as it's all book-learned knowledge.
> I can't parse what you mean by this.
The point is that humans care about the state of a distributed shared world model and use language to perform partial updates to it according to their preferences about that state.
Humans who prefer one state (the earth is flat) do not -- as a rule -- use language to undermine it. Flat earthers don't tell you all the reasons the earth cannot be flat.
But even further than this, humans also have complex meta-preferences of the state, and their use of language reflects those too. Your example is relevant here:
> My dad absolutely insisted that the water draining in toilets or sinks are meaningfully influenced by the Coriolis effect [...]
> [...] should have been able to figure out from first principles why the Coriolis effect is exactly zero on the equator itself, didn't.
This is an exemplar of human behavior. Humans act like this. LLMs don't. If your dad did figure out from first principles and expressed it and continued insisting the position, I would suspect them of being an LLM, because that's how LLMs 'communicate'.
Now that the what is clear -- why? Humans experience social missteps like that as part of the loss surface. Being caught in a lie sucks, so people learn to not lie or be better at it. That and a million other tiny aspects of how humans use language in an overarching social context.
The loss surface that LLMs see doesn't have that feedback except in the long tail of doing Regularized General Document Corpora prediction perfectly. But it's so far away compared to just training on the social signal, where honesty is immediately available as a solution and is established very early in training instead of at the limit of low loss.
How humans learn (embedded in a social context from day one) is very effective at teaching foundational abilities fast. Natural selection cooked hard. LLM training recipes do not compare, they're just worse in so many different ways.
https://www.anthropic.com/research/introspection
Its hard to tell sometimes because we specifically train them to believe they don't.
I don't think the version of self awareness they demonstrated is synonymous with subjective experience. But same thing can be said about any human other then me.
Damn, just let me believe all brains are magical or I'll fall into solipsism.
> If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.
Google "strawman".