Edit: What is happening with people these days? They seem to be reducing people and their minds to machines more easily than ever.
No it isn't, neural networks aren't a network of neurons, they are just named after neurons but they are nothing alike. Neurons grow new connections etc and do a whole slew of different things that neural networks doesn't do.
The ability to grow new connections seems like an integral part to intelligence that neural networks can't replicate, at least not without changing them to something very different than they are today.
Forming new connections Is not integral. Connections don’t form in seconds so second to second existence is a static neural network. Thus your second to second existence is comparable to an LLM in the sense that both neural networks don’t form new connections.
We can though. We can make neural networks form new connections.
The brain neurons are not the same kind of object as neural net neurons, hence its not a neural net. It is like saying a dragonfly is a helicopter since the helicopter were designed to look a bit like a firefly, no a dragonfly isn't a helicopter they fly in totally different ways the dragonfly has wings and thus don't even move similarly.
> Connections don’t form in seconds so second to second existence is a static neural network
But a human isn't intelligent over a second, and synapses do form over longer periods which is what it takes for humans to solve harder problems.
> We can though. We can make neural networks form new connections.
I said the neural network couldn't form those, not that we couldn't alter the neural network. That is a critical difference, our brains updates itself intelligently on its own, that is integral to its function, if our brains couldn't do that we wouldn't be intelligent, when that happens we say a person has Alzheimer's. You wouldn't hire a person with far gone Alzheimer's precisely because they can't learn.
Edit: So you can see in brains being able to form connections makes it smarter. In our neural nets when we try to alter their connections live they get dumber, which is why we don't do it. That should tell you that we are clearly missing something critical here.
So you’re saying as you talk to me right now you’re not conscious? Neurons didn’t grow connections right now so you’re not alive? You won’t form connections in an hour so during that hour your some kind of robot and suddenly when you form a connection you’re not?
We have algorithms that can self evolve neural networks in computers on its own. So you’re actually wrong here there’s progress on this front.
I am not missing anything critical. I know there are huge differences between the human brain and an LLM but the statement remains true. Both are neural networks. The problem here is that you either don’t understand vocabulary or you think everyone around you is so dumb they can’t see the differences.
Everyone and I mean everyone knows what you’re saying in the paragraph above. It’s common knowledge anyone can pull out their ass and your regurgitating it as if your a scientist or expert. Get this, BOTH are still neural networks and both have commonalities despite the distinctions. Both are also Turing complete.
Fun fact. Neural nets are not made of neurons and more than the python programming language is made of snakes.
The neuron in ML is based off of the simplest mathematical model of what a neuron does. The neuron in a human brain is a physical realization of this with much much more complexity where much of this complexity is arbitrary as a side effect of being physically realized.
But again I’m regurgitating what everyone knows
You can't build a model out of an object and then say that said object is an instance of the model. The model is an useful derived abstraction that sometines can serve for better understanding reality and making predictions (emphasis in sometimes), but it is not reality itself.
Appearances can be deceptive and intelligence in brains may be more complex than they currently looks, but appearances are also the only thing most of us can judge it by at this point — while there's more stuff going on in living cells, nothing I've seen says the stuff needed to keep the cells themselves alive is directly contributing to the intelligence of the wet network in my skull.
It's quite surprising that such a simple mind as an LLM is so very capable. What is all the rest of our human brain doing, when the LLMs demonstrate that a mere cubic centimetre of brain tissue could be reorganised to speak 50 languages fluently, let alone all the other things?
Plus, sure, LLMs are simpler (that only works if you don't count the full complexity of its substrate: a lot of GPUs, interconnects, Data Centers, etc), yet they consume an insane amount of energy and matter when compared to humans. It is surprising that LLMs are so capable, but for 2000kcal a day humans are still more impressive to me.
People see a model dominating language and start thinking the only thing humans do is high-level language and reasoning. That may be why the brilliant minds of this decade talk about replacing white-collars, but no plate-cleaning bot in sight.
Metacommentary: a lot of LLM-human equality apologists seem to have little knowledge of the AI field and its history, as these themes are not new at all.
These days, so do the AI (even though these ones are still called LLMs, which I think is now a bad name even though I continue to use the term without always being mindful of this).
> I've yet to see an humanoid robot controlled by an LLM that can tackle difficult physical problems like riding a bike or skiing in for the first time real-time, just as humans, that need to coordinate at least 5 senses (some say up to 20 but I digress) plus emotions and a train of thought to do so.
1) I'd agree that AI are slow learners (as measured by number of examples rather than wall clock). For me, this is more significant than the difference in wattage.
2) 13 years ago, before the Transformer model: https://www.youtube.com/watch?v=mT3vfSQePcs
3) We don't use 5 senses to ride a bike; I can believe touch, vision, proprioception, and balance are all involved, but that's only 4. Why would hearing, smell, or taste be involved?
For emotion, my weakly-held belief is that this is what motivates us and tells us what the concept of "good" even is — i.e. it doesn't give us reasoning beyond being a motivation to learn to reason.
> It is surprising that LLMs are so capable, but for 2000kcal a day humans are still more impressive to me.
Our biological energy efficiency sure is nice, though progress with silicon is continuing even if the rate of improvement is decreasing: https://en.wikipedia.org/wiki/Koomey%27s_law
That said, I don't consider the substrate efficiency to be an indication of if the thing running on it is or isn't "reasoning" — if the same silicon was running a quantum chemistry simulation of the human brain it would be an even bigger power hog and yet I would say it "must be" reasoning, and conversely we use phrases such "turn your brain off and enjoy" to describe categories of film where the plot holes become pot holes if you stop to think about them (the novel I'm trying to write is due to me being nerd-sniped in this way by everything wrong with Independence Day).
> That may be why the brilliant minds of this decade talk about replacing white-collars, but no plate-cleaning bot in sight.
I mean, I've got a dishwasher already… ;)
But more seriously, even though I'm getting cynical about press releases that turn out to be smoke and mirrors, (humanoid) robots doing housework is "in sight" in at least the same kind of way as self driving cars (which has been a decade of "next year honest" mirages, so I'm not giving a timeline): https://www.youtube.com/watch?v=Sq1QZB5baNw
> Metacommentary: a lot of LLM-human equality apologists seem to have little knowledge of the AI field and its history, as these themes are not new at all.
I've been feeling the same, but on both sides, pro and con. The Turing paper laid out all the same talking points I've been seeing over the last few years, and those talking points weren't new when he wrote them down.
2. When on a bike you need to be full alert and hearing all the time or you might provoke an accident. Sure you can use headphones, but it is not recommended. Also, that robot you showed is narrow AI, it should be out of discussion when arguing about complex end-to-end models that are supposedly comparable to humans. If not, we could just code any missing capability as a tool, but that would automatically invalidate the "llms reason/think/process the same way as humans" argument.
3. Agree, I expect the same in the future. I'm talking about the present though.
4. ;) a robot that helps with daily chores can't come soon enough. More important than coding AI for me.
PS: Not sure if we're talking about the same argument anymore, I'm following the line of "LLMs work/reason the same as humans" not "AI can't perform as good as humans (now and in the future)"
Yeah, it's hard to keep track :)
I think we're broadly on the same page with all of this, though. (Even if I might be a little more optimistic on tokenised processing of images).
I've found I work differently. Some technical problems like architecture can only be solved by aligning my emotions with the problem, then my primal brains find a good solution that matches my emotional wants.
But when my emotions doesn't care about the problem my intuition stops working to solve it, and then I can't find any solution.
Why would emotions be needed to solve problems? Because without emotions your can't navigate a complex solution space to find a good solution, its impossible. Meaning if we want an AI to make good architecture for example, we would need to make the AI have feelings about architecture such as what setups are good in different situation etc. Without those feelings the AI wouldn't be able to solve the problem well.
Then after you have came up with a solution using emotions you can then verify it using logic, but you wouldn't have came up with that solution in the first place without using your emotions. Hence emotions are probably needed for truly intelligent reasoning, as otherwise you can't properly guide yourself through complex solution spaces.
You could code something that fills the same role and call it something different than emotions, but if it works the same way and fills the same functions its still emotions.
(Using this definition then AlphaGo uses emotions to prune board states to check, stuff like that, it doesn't know those states are best to check, its just a feeling it has)
The fundamental mathematical concept of a neuron is what both neural networks operate on in the end.
Be specific.
And no, it doesn't "happen with every new technology." Nothing even remotely like this has been seen before the present decade.
Be able to express language with a training/knowledge/etc corpus of far smaller size.
Plus even when humans are reading books a mental image of what's going on in the story is common. Not everyone has that, but it shows how much a basic LLM lacks and multi-modal would add.
Now the real question to my mind is whether we can train models with actual empathy to learn from the experiences of other people without having to go through the experience directly. Doing so would put them above many individuals' understanding already. . .
> Be able to express language with a training/knowledge/etc corpus of far smaller size.
Nobody knows what it would mean to test for:
> Have experiences and feelings.
And I am unclear if:
> Have personal knowledge of something.
Would or would not be met by putting them in charge of a robot of some kind to collect experiences?
I'd certainly agree that the default is to make them "book smart" rather than "street smart", but at most that's an "is not" rather than "cannot".
Edit: Ok, let me be less harsh and play with your argument. Can an LLM have an acid/psilocybin/ketamine trip, distort its view of self and reality, then rewire some of its internal connections and come a little different based on the experience? I guess not, there are more examples and they all show that as far as we know LLMs are not minds/brains and viceversa even if they seem similar in a chat interface. (If you don't empathise with that example remove the drugs and change them for a near-death experience).
I strongly argue humans are not (just) LLMs, but I think we're far from getting evidence*. I think we will get to AGI before we know how the brain works and there is no dicotomy in that, if anything the tech is showing us that we can get at least some intelligence in other substrates.
*PS: fwiw ausence of evidence does not support any sides. I shouldn't remark that, but here we are.
This, while true, is putting the cart before the horse.
One needs to define what the question is before it is possible to seek evidence.
I see many things different between Transformer models and brains, but I do not know which, if any, matters.
> Can an LLM have an acid/psilocybin/ketamine trip, distort its view of self and reality, then rewire some of its internal connections and come a little different based on the experience?
Specifically those chemicals? No, not even during training when the weights aren't frozen, as nobody bothered to simulate the relevant receptors*.
Can LLMs experience anything, in the way we mean it? Nobody even knows. We don't know what the question means well enough to more than merely guess what to look for.
Do the inputs to an LLM, a broader idea of "experience" without needing to solve the question of which definition of consciousness everyone should be using, rewire some of its internal connections? All the time.
* caveat: it doesn't, at first glance, seem totally impossible that the structure of these models is sufficiently complicated for them to create the simulation of those things in the weights themselves in order to better model the behaviour of humans generating text from experiencing those things. But I also don't have any positive reason to expect this any more than I would expect an actor portraying a near-death-experience to have any idea what that's like on the inside.
I won't continue with these discussions as I feel people are being just obtuse about this and it is becoming more emotional than rational.
With a lack of precision necessary when the phrase "You need scientific evidence" is invoked.
It's very easy to trip over "common sense" reasoning and definitions — even Newton did so with calculus, and Euclid did so with geometry.
> I won't continue with these discussions as I feel people are being just obtuse about this and it is becoming more emotional than rational.
As you wish. I am relatively unemotional on this.
Can these models do that?
Can *a* brain do this?
Every effort I've seen has (1) involved *many people* working on different parts of the problem, and (2) so far we've only got partial solutions.
Even if you allow collaboration, then when (2) goes away, your question is trivially answered "yes" because we just feed into the model the same experiences that were fed into the human(s) who invented the model, and the simulation then simulates them inventing the same model.
These models doesn't work that way. Its probably possible to make such a model, but currently we don't know how to make them.
I didn't ask for it, one of the Phi models got confused part way though and started giving me a python script for machine learning instead of what I actually asked for.
I'm saying I have no evidence that humans have ever demonstrated this capability either, and if we were to invent such an algorithm and model (even more generally than LLM) the AI that is this would by definition necessarily have to pass this test.
- neurons have signal surpression (-ive activation) as well as propagation (+activation)
- neurons appear to process and propagation due to unknown internal mechanisms.
- there are complex sub structures within biological neural nets where the architecture is radically different from in other sections of the network, strongly in contrast to the homogenous structure of ANN's
- many different types of neurons with different properties and behaviors in terms of network formation and network activity are present in BNN's in contrast to ANN's
- BNN's learn during processing. ANN's are static after training.
I'd wager that future models (which admittedly may not look much like today's LLMs) will blur the lines between mutable context and immutable weights. When that happens, all bets are truly off.
"may not look much like today's LLMs" is really sweeping the whole point under the rug. The difference between what we are doing now with static LLM models and dynamic models will require algorithmic changes that have not yet been invented. That's in addition to the fact that the processing of a single neuron is completely parallel with respect to inputs. GPUs make them more parallel, but the analog devices we are trying to mimic are vastly more complicated than what we are using as a substitute.
Either that, or it's witchcraft.
(Either that, or somebody with both time and talent to waste is taking the piss and pretending to be an AI.)
If anyone ever does succeed in updating weights dynamically on a continuous basis, things are going to get really interesting really quickly. Even now, arguments based on relative complexity are completely invalid. A few fast neurons are at least as good at 'thinking' as a lot of slow ones are.
The writing is still derivative meandering pulp. No need to invoke magic. It's cool that it can make complete paragraphs though. LLMs are a huge breakthrough. They're just not intelligent.
Updating weights on a continual basis requires processing weights productively on a regular basis. That's many flops per weight away from where we are in processing and bandwidth. The computation limitations still apply.
As I said, neuroscience be damned, we've got transformers now. (/s)
"My king, the Hittites have created scrolls that are wide and long despite that we burned their caravan taking palm leaves to the north. They have found a way to flense lambs to make pages themselves, and their ink is pure silver. They will soon have an army of Gollems and will destroy your empire. To stop this all I need is ten bags of gold." - Sargon the scribe, near Baghdad second month, 2024 BC
"My king, alas all the gold has gone, and yet neither will the gollems walk, nor have the Hittities come. My dancing girls grow cold, and I have no figs for my table all I need is 20 bags of gold" - Sargon the scribe, near Baghdad third month, 2024 BC
What happened to Sargon next is not recorded...