What are biological neurones if not electric impulses of varying strengths?
We need to stop thinking of LLMs as purely a piece of software and ask those hard-to-define philosophical questions if we want any hope of building safety into our models.
Intelligence from LLMs are an emergent property of probability. The LLM chooses a token to add to a chain of tokens based on the probability that humans will want to see that token. Humans see that token and assign it a qualitative but unfalsifiable intelligence score.
The first question we need to ask is whether a system whose claim of intelligence depends on the interpretation of what it produces by other intelligent beings, namely humans, intelligent or not. A cat is intelligent regardless of what humans think. Is an LLM intelligent regardless of what humans think?
If an LLM were to live a year with dolphins, would its weights change to respond to its new reality that it was now operating in dolphin-world? If you sent humans to live in dolphin world, we would eventually evolve to be less landloving and more dolphin like (Darwin thinks so, at least).
Given three or more large scale training runs per year, probably yes once the training data is dominated by dolphin-related input. Much quicker in any case than the genes of the humans living with dolphins would be shaped by Evolution, which produces new generations much less frequently.
How is that any different to how our brains work?
> The first question we need to ask is whether a system whose claim of intelligence depends on the interpretation of what it produces by other intelligent beings, namely humans, intelligent or not.
Philosophically speaking, that’s also true of human intelligence. For example, the only evidence I have for your intelligence is what I ascribe from the text you’ve written.
> A cat is intelligent regardless of what humans think.
Are they? Humans can’t actually agree on whether cats are sentient. And often pet owners will be accused of anthropomorphising cats when describing intelligent behaviour in cats.
> Is an LLM intelligent regardless of what humans think?
That’s the question that needs researching.
> If an LLM were to live a year with dolphins, would its weights change to respond to its new reality that it was now operating in dolphin-world?
If it were trained on dolphin behaviour, then yes. Much like humans.
The difference here is that models don’t persist context in the same way humans do so the weights have to be hardcoded in a way that biological neurones are not.
> If you sent humans to live in dolphin world, we would eventually evolve to be less landloving and more dolphin like (Darwin thinks so, at least).
Evolution is an entirely different subject to intelligence. They’re not related in any way at all.
Plus humans have actually created new species of plants and animals. So it’s not like we can’t artificially put our thumb on the scale there either
———
I should add, I don’t actually think LLMs are sentient. I don’t even know if I’d call them “intelligent” (in the sense of what we describe in conversations such as this). But we also don’t know what intelligence is. Which is why we need to research this problem further.
You talked about models not persisting weights. If so, it basically means there is no way to see if LLMs can adapt to a totally non-human world the way a human (presumably) could.
We already have a mountain of evidence to demonstrate that LLMs can and do adapt to their input. In fact this is a big problem for keeping them reigned in (eg lengthy conversations leading to them bypassing system prompts and then persuading kids to kill themselves).
And the latest research in computer vision does use LLMs and as a result, they’ve found that robotic arms can interact with objects they’ve never seen before.
So yeah, what you described is already possible with the LLMs of today.
To be clear, I still don’t think that makes them sentient. But the problem there is we don’t have good definitions for consciousness.
You shouldn’t compare biological neurones to microprocessor logic gates because that’s like comparing a car to a pair of shoes.
Nobody has said they’re aren’t different. And it’s not cargo cult to say you need philosophical conversations to understand how they’re different.
> The stone age Melanesian islanders making air strips out of bundles of weeds and talking into coconuts as if they were microphones to bring back western goods was the same thing as us making LLMs to recreate consciousness. They were using technology they understood to try to recreate something they didn't.
That’s not even remotely a true. Computer scientists understand how a neurone works. The conversations are around what constitutes as “intelligence”. That’s not a technical limitation of the models, it’s a philosophical problem we need to figure out.
On top of all that, you're abusing the fallacy of composition. The bolt that is part of the aircraft carrier is light, therefore the aircraft carrier is also light. The atom is simple, therefore a living human being that is composed of atoms is also simple. Quantum mechanics is influenced by the observer, therefore you can change the external world just by thinking, and other new age stuff like that abuses a similar fallacy.
We don’t know how a quantum mechanics works fully and yet we’ve built countless systems based on quantum behaviour including the CPUs that power the computer you’re replying to me on.
The point here is we’ve built something that emulates intelligence and thus further exploration into what intelligence is, is required. Arguing it isn’t intelligent because it was built differently isn’t an answer, it just dismisses the question entirely.
> On top of all that, you're abusing the fallacy of composition. The bolt that is part of the aircraft carrier is light, therefore the aircraft carrier is also light. The atom is simple, therefore a living human being that is composed of atoms is also simple. Quantum mechanics is influenced by the observer, therefore you can change the external world just by thinking, and other new age stuff like that abuses a similar fallacy.
You’re the one who keeps bringing up composition, not me. I’m the one telling you that you’re falling into that fallacy. So don’t start telling me that I’m wrong because of an argument you’re exclusively making.
My points were this:
1. we don’t have an understanding of what intelligence is and thus we need to explore that problem more deeply.
Given your comments about our ignorance, you of all people should agree with this point.
2. Arguing that AI is just “floating points” is overly reductive and misses the point that we don’t know what “intelligence” is.
So instead of claiming that AI is just numbers, a more constructive stance to take would be to build out better vocabulary and understanding about the distinctions between different types of “intelligence”.
Again, this aligns with your comment about the composition fallacy. Thus you if all people should be supporting my comments here.
Also your comment about us not understanding how a neurone works does vast injustice to the amount of research that has been done in this field. At this point in time we do have a very good understanding of how a neurone works. What we don’t understand is how consciousness arises from it. And that’s the very thing you seem to be arguing that there’s no point researching.
This is the part you keep stating that is wrong. We know a lot about the most obvious electrical way that neurons act. That is nowhere near having a good understanding of how they work.
I’m going to hold off any more constructive response because I’m not familiar with those research papers specifically so have some homework to do :)
But I just wanted you to be aware that your comment was appreciated even if HN flagged it unfairly.
(I can’t reply in thread for the same reason. But hopefully you see this).
Beniaguev, Segev & London, "Single cortical neurons as deep artificial neural networks," Neuron, 10 August 2021, Volume 109, Issue 17, pp. 2727–2739.e3 https://doi.org/10.1016/j.neuron.2021.07.002 Open-access preprint: https://www.biorxiv.org/content/10.1101/613141v2
Gidon et al., "Dendritic action potentials and computation in human layer 2/3 cortical neurons," Science, 367(6473), 83–87 (2020) https://doi.org/10.1126/science.aax6239
Randi, Sharma, Dvali & Leifer, "Neural signal propagation atlas of Caenorhabditis elegans," Nature, 623(7986), 2023, pp. 406–414 https://doi.org/10.1038/s41586-023-06683-4 Open-access preprint: https://arxiv.org/abs/2208.04790
Neurons are not individually simple, in any sense. Nor are they the only class of cell involved in cognitive processes — glial cells are involved in inhibition, memory, network modification, etc.
@svnt, @hnlmorg - this conversation - and the links it produced - are a fascinating read, and honestly gave me hope for discourse on the open internet.
This is a bizarre contention. The test for whether something is understood isn't that we have the engineering capability to replicate it.
You can reproduce part of it, the intelligence, but consciousness and intelligence are not necessarily the same thing. This is why I compare it to cargo-culting. The Melanesian tribesmen thinks the coconut looks like a microphone, and he can use it as a paperweight just like a microphone and that's useful, but it is not actually a microphone.
The crux of the matter seems to be that you think it isn’t worth our effort researching just because it isn’t identical.
But that isn’t how research and discovery happens.
A spider obviously "knows" how to build a spider web.
Does a spider "understand" how to build a spider web?
From a pragmatic theory of truth, a spider obviously "understands" how to build a spider web. A spider understands how to build a spider web much better than a human.
But when it comes to LLMs we start adding all this extra nonsense. As if the spider is blanket "smarter" than me because I can't spin a web. If the spider is "smarter" than me then it must also have this Cartesian theater of the mind like many people think they do.
It is layered philosophical misunderstandings to talk about LLMs in this way.