Is my toddler a stochastic parrot?
newyorker.com
newyorker.com
Perhaps worth noting that it is possible to teach infants (often starting at around 9 months) sign language so that they can more easily signal their desires.
Some priority recommended words would probably be:
* hungry/more
* enough/all done (for when they're full)
* drink (perhaps both milk/formula and water† gestures)
See:
* https://babysignlanguage.com/chart/
* https://www.thebump.com/a/how-to-teach-baby-sign-language
These are not (AFAICT) 'special' symbols for babies, but the regular ASL gestures for the work in question. If you're not native English-speaking you'd look up the gestures in your specific region/language's sign language:
* https://en.wikipedia.org/wiki/List_of_sign_languages
* https://en.wikipedia.org/wiki/Sign_language
† Another handy trick I've run across: have different coloured containers for milk and water, and consistently put the same contents in each one. That way the infant learns to grab a particular colour depending on what they're feeling like.
My favourite moment was in March, my daughter was about to turn 2 and wasn't speaking yet.
I asked her if she would like to hear some music.
She made the sign for dog.
I searched youtube for some songs about dogs and she shook her head.
She made the sign for tree.
I was like "dog, tree", she nodded. Hmmm...
I was searching for "dog tree music" when one of the pictures that came up was a christmas tree.
She pointed to that excitedly!
I was like "dog christmas tree music" ... it took me a second to realise that she wanted to listen to the Charlie Brown Christmas soundtrack that I had had playing off YouTube at Christmas 3 months previously!
I put that on again and we danced around to it.
I thought that was totally wild! It was the first time I remember her communicating a really sophisticated preference other than just wanting to eat/drink/help etc.
ML is pretty long way from being able to make generalizations like that from…20 samples?
Settlers interacting with natives knew full well how complex their desires were – they lived side-by-side, traded, socialised, and learned from each other.[1] Any suggestion of a primitive native is self-comforting propaganda from the industrial complex that comes after the settlers.
[1]: Indians, Settlers, and Slaves in a Frontier Exchange Economy; Usner; Omohundro Institute; 2014.
It took very minimal effort on our part, and was very rewarding for him; certainly a lot better than him crying with the hope that we could guess what he wanted. Definitely recommended for any new parents.
The best moment was when he was sitting on the floor, and looked up at his mom and made the "together" sign, it was heart melting.
I have friends who had much more success with it, but the value will largely depend on your child’s relative developmental strengths. A friend’s son with autism got literally years’ benefit out of the gestures before verbal speech caught up.
Could still useful: instead of shouting across the playground on whether they have to go potty you can simply make the gesture with minimal embarrassment. :)
Please don't blame this on the signs! This doesn't mean that he would have learned to speak earlier if not for the signs. I'd be glad that he could communicate proficiently at all.
were you always talking when you signed to them? maybe they thought it went together.
At about the same age, I bought my son a mango lassi. He looked suspiciously at it, but took a sip. With a look of shocked delight he tilted it back, back, back, and emptied the cup!
Then he put it down, looked at me, and said, "Want more!"
I'm looking forward to kids out of the house. But there are some moments that I treasure.
It probably doesn't matter either way for babies, but fyi ASL isn't a sign version of English; it is its own language. In fact American Sign Language is more closely related to French Sign Language than to British Sign Language. The Australian and New Zealand Sign Languages are largely derived from British Sign Language, so there isn't really a correlation between English speaking regions and ASL. Canadians mostly use American Sign Language and French Canadian Sign Language.
For example, voluntary supination/pronation of the forearm is generally not something a 9month old can do. If you try and teach them a common sign for "enough/finished" (fist closed, thumb pointed out, then rotation of the forearm back and forth), or "done" and "more" in the parent link, they probably won't be able to do it properly. They can copy something close to that (thumb out and wobbling their hand around? good enough!) so you have to go with the flow.
There are quite a few signs like that actually, so try and think about how many muscles move together, and how controlled or complex that is. Simple stuff is good -- and doable.
One of my favourite memories of that was when we went to see the Vivid light show in Sydney and there was a contortionist on the street so we stopped to watch. I looked into the stroller and said "What do you think?" and she made the sign for "finished". So we moved on.
This is interesting. My son also did "index finger up" in response to "thumbs up" for the longest time. Why is the thumb so hard to manipulate? Late addition to the evolutionary sequence?
I wish I'd learnt sign language before having kids so I just already knew how to do it, it's so cool. Props to the Ms. Rachel videos for including so many signs.
I can't remember exact ages and timeframes-- that time of my life is "blurry". I wish I could remember all the gestures we used. (The only ones I can remember now are "milk", "apple", and "thank you".) As she became verbal she quickly transitioned away from them.
-She learned some gestures of mine instead, which I didn't realize I was doing.
-Defaulted to speech as soon as possible because it was just easier.
But I can never seem to find more information about it, and can't recall the documentary name for the life of me. Since you seem to have at least some familiarity with this, do you happen to know what I'm talking about?
I think it makes a big difference, both for us (parents, to have a clue what they want) and the kid (being understood, when they can’t speak).
Almost 20 years later, I still know all the signs for baby food ;)
You can teach chatgpt too as well. It's like a toddler. A very articulate toddler:
https://chat.openai.com/share/40c94561-2505-4938-8331-7d10ae...
It makes mistakes as any human baby would. And as a parent you can correct it.
All this means is that learning an arbitrary sign language isn't a differentiator.
Which BTW I don't think is a completely absurd comparison, see https://mattasher.substack.com/p/ais-killer-app
I point this out because I think the idea of comparing ourselves to recent tech is more about using the technology as a metaphor for self, and it's worth incorporating the other ways we have done so historically for context.
[1]: https://online.ucpress.edu/SLA/article/2/4/542/83344/The-Bra...
Its way more productive and way less woo-woo to understand that humans have a certain tendency towards comparison, and we tend to create things that reflect our current values and conceptions of ourselves. And that "technological progress" is not a straight line, but a labyrinthine route that traces societal conceptions and priorities.
The desire for the llm to be like us is probably more realistically our desire to be like the llm!
But then again, an apple is nothing like an orange, particularly if you want to make an apple pie.
The purpose of a comparison is important in helping to define its scope.
Step B: turn it on its head "the human brain is nothing more than... <insert machine here.>"
It's a bit tautological.
The worry is that there's a Step C: Humans actually start to behave as simple as said machine.
A very large number, e.g., lots of implants and prosthetic devices for one fairly large class.
They emulate the faculty, rather than biology.
Neural networks are not based on how neurons work. They do not copy aspects of us. They call them neural networks because they are sort of conceptually like networks of neurons in the brain but they’re so different as to make false the statement that they are based on neurons.
"Behold! The Mechanical Leg! The first technology that actually copies aspects of our very selves! Think of what wonders of self-discovery it shall reveal!" :p
P.S.: "My god, it is stronger in compression rather than shear-stresses, how eerily similar to real legs! We're on to something here!"
Both Humans and GPT are neural networks. Who cares that GPT doesn't have feathers or flap its wings? That's not the question to care bout. We are interested in whether GPT flies. You can sigh to Kingdom come and nothing will change that.
We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse, so we stuck with backpropagation. We've made more complicated neurons that better resemble biological neurons, but it is faster and works better if you just add extra simple neurons, so we do that instead. Spiking neural networks have connection patterns more similar to what you see in the brain, but they learn slower and are tougher to work with than regular layered neural networks, so we use layered neural networks instead.
The Wright brothers probably experimented with gluing feathers onto their gliders, but eventually decided it wasn’t worth the effort. Because that's not what is important.
I recommend reading Synaptic Organization of the Brain or getting into if you are brave, the primary literature on retinal processing of visual input.
https://cs.stanford.edu/people/eroberts/courses/soco/project....
Key word: "neurophysiologist"
Consciousness is first a word and second a concept. And it’s a word that ChatGPT or Llama can use in an English sentence better than billions of humans worldwide. The software folks have made even more progress than sociologists, psychologists and neuroscientists to be able to create an artificial language cortex before we understand our biological mind comprehensively.
If you wait until conscious sentient AI is here to make your opinions known about the implications and correct policy decisions, you will already be too late to have an input. ChatGPT can already tell you a lot about itself (showing awareness) and will gladly walk you through its “thinking” if you ask politely. Given that it contains a huge amount of data about Homo sapiens and its ability to emulate intelligent conversation, you could even call it Sapient.
Having any kind of semantic argument over this is futile because a character AI that is hypnotized to think it is conscious, self-aware and sentient in its emulation of feelings and emotion would destroy most people in a semantics debate.
The field of philosophy is already ripe with ideas from hundreds of years ago that an artificial intelligence can use against people in debates of free will, self-determination and the nature of existence. This isn’t the battle to pick.
My point is that there are good reasons to believe that a hypothetical LLM that can pass a Turing test is a philosophical zombie, i.e. it can mimic human behavior but doesn't have an internal life, feelings, emotions, and isn't conscious. Whether that distinction is important is another question. LLMs provide evidence that consciousness may not be necessary to create sophisticated AIs that can pass or exceed human performance.
I'm curious how you know this. Certainly an LLM doesn't have human internal life, but to claim it has no internal life exceeds our state of knowledge on these topics. We simply lack a mechanistic model of qualia from which we can draw such conclusions.
Once you get that out of the way, sure, I guess it could be "alive" per the same loose definition that any electrical system can exist in an actuated state. It's software, though. I don't think it's profound or overly confident to say that we very clearly know these systems are inanimate and non-living.
>An LLM is math
Isn't everything math? Math is what we use to describe and model the nature of our reality.
>Those things aren't alive
We don't have a universal definition of alive, or at least I'm not aware of one that isn't purely philosophical or artistic. We therefore can't actually determine what is and isn't alive with much confidence.
>both the software and hardware used to facilitate it is artificial
Define artificial? We're made of exactly the same stuff. Complex chemical elements that were created from more basic ones in collapsing stars, arranged in a particular configuration with electricity flowing through it all and providing some kind of coordination. We're definitely a bit more "squishy", I'll give you that.
I'm also not even sure why you're talking about "alive" or "inanimate". From context it's clear we're talking about "internal life", meaning experience, feelings, thoughts, etc., not some ability to procreate, or even necessarily some continuity of existence.
Then there's the concept of panpsychism.
That's what panpsychists believe. Regardless, this is a silly comparison. Show me a rock that can carry an intelligent conversation and then you might have a point.
In a sense, yes! But to understand that you will first have to precisely define "pain". Good luck.
Regarding this line:
> ChatGPT can already tell you a lot about itself (showing awareness) and will gladly walk you through its “thinking” if you ask politely.
Is it actually walking you through its thinking? Or is it walking you through an imagined line of thinking?
Regardless, your main point still stands. That a program doesn't think the same way a human does, doesn't mean it isn't "thinking".
You can prompt an LLM model to provide reasoning first and an answer second and it becomes one and the same.
Worth keeping in mind that all of these points are orthogonal to the quality of reasoning, the bias, or the intentions of the system builders. And building something that emulates humans convincingly, you can expect it to emulate both the good and bad qualities naturally.
Because a human is measuring it unfairly.
The output without CoT is valid. It is syntactically valid. The observer is unhappy with the semantic validity, because the observer has seen syntactic validity and assumed that semantic validity is a given.
Like it would if the model was alive.
This is observer error, not model error.
I think the whole point the article is making, though, is that overvaluing LLMs and their supposed intelligence because they excel at this one axis of cognition (if you’re spicy) or mashed-up language ability (if you’re a skeptic) doesn’t make sense when you consider children, who are not remotely capable of what LLMs can do, but are clearly cognizant and sentient. The whole point is that those people—whether they can write an essay or not, whether they can use the word “consciousness” or not—are still fundamentally alive because we share a grounded, lived, multi-sensory, social reality.
And ChatGPT does not, not in the same way. Anything that it expresses now is only a mimicry of that. And if it eventually does have its own experiences through embodied AI, I’d be interested to see what it produces from its own “life” so to speak, but LLMs do not have that.
I’m not saying computers can’t have something like it, but it would be so fundamentally different as to be completely alien and unrelatable to humans, and thus (IMO) non-contiguous with human culture.
I wish we could keep this as an immutable truth, but give some sick girls and boys in Silicon Valley a few more years and they will make true creatures. No, I think that we should be honest to ourselves and stop searching for what make us special (other than our history, and being first, that is). It's okay to be self-interested and say "we want to remain in control, AIs should not be, no matter how much (or if) better they are than us."
Thank you for solving the riddle!
What AI lacks compared to humans, is what humans share in common with all of life, a motivation for survival.
As the creators of AI we need to be mindfully aware that we are the moral agents, we are the ones with motives, desires and goals, and we are responsible for making ethical decisions.
My bicycle can go down a hill much faster than my feet, but it doesn’t go anywhere without me because it doesn’t have anywhere it wants to go. I’m the one providing directions.
Uh, wouldn't that apply equally to any other topic that has been argued extensively before, and to any tenable position on those topics? Like, I can make ChatGPT argue against your LLM sentience apologist-bot just as easily.
However, it's a bit annoying that the focus of the AI anxiety is how AI is replacing us and the resolution is that we embrace our humanity. Fair enough, but at least to me the main focus in my AI anxiety is that it will take my job - honestly don't really care about it doing my shitty art.
An enlightened Humanity could solve this by separating the income from the job, but we live in a Malthusian Darwinian world where growth is paramount, "enough" does not exist, and we all have to justify and earn our living.
I really like programming to fix things. Even if I weren’t paid for it, even if I were to win the lottery, I would want to write software that solved problems for people. It is a nice way to spend my days, and I love feeling useful when it works.
I would be very bummed - perhaps existentially so - if there were no practical reason ever to write software again.
And I know the same is true for many artists, writers, lawyers, and so on.
I'm not too concerned about that being a reality in our lifetimes though.
And I think about that a lot.
Right now, we all own "robots" who spellcheck our words (editor) , research (librarian), play music (musican), send messages (courier), etc.
All these jobs are "lost", but at the same time, we wouldn't have had money to pay these many employees to live in our pocket.
I think it was clear about how language is not thought.
That leads to the intrinsic realization that our physical existence, our observance of reality is what is critical.
Also, AI taking jobs at scale is unlikely, the only place its going to do anything is low level spam and content generation.
For anything which has to be factual, its going to need humans.
Now: a chance encounter with someone of a different faith leads a citizen to respect the religious freedom of others in the realm of self-determination.
Future: a young hacker's formative experience leads to the idea that citizens should have the basic right to change out their device's recommendation engine with a random number generator at will.
Those future humans will still think of themselves as exceptional because the AI tools will have developed right alongside the current human-exceptionalist ideology.
Kinda like those old conservative couples I see in the South where the man is ostensibly the story teller and head of household. But if you listen long enough you notice his wife is whispering nearly every detail of importance to help him maintain coherence.
You've never actually seen this happen because it doesn't happen. This is not how real people interact, it's how the caricatures in your head interact.
The concept of consciousness is wildly more expansive and diverse than computation, rather than the other way around. A strict materialist account or "explanation" of consciousness seems to just end up a category error.
I take it as no surprise that a website devoted to the computers and software often insists that this is the only way to look at it, but there are entire philosophical movements that have developed fascinating accounts of consciousness that are far from strict materialism, nor are they "spiritual" or religious which is a common rejoinder by materialists, an example is the implications of Ludwig Wittgenstein's work from Philosophical Investigations and his analysis of language. And even in neuroscience there is far from complete agreement on the topic at all.
if there is no duality, then what else can it be?
> there are entire philosophical movements that have developed fascinating accounts of consciousness that are far from strict materialism
philosophers can bloviate all day. meanwhile the computer nerds built a machine that can engage in creative conversation, pass physician and bar exams, create art, music and code.
I'm not saying there's nothing to learn from philosophy, but gee you have to admit that philosophers come up a little short wrt practical applications of their "Philosophical Investigations"
Personally these sorts of things don't really matter to me- I don't really care if other people are conscious, and I don't think I could prove it either way- I just assume other people are conscious, and that we can make computers that are conscious.
And that's exactly what I'm pushing for: ML that passes every Turing-style test that we can come up with. Because, as they say "if you can't tell, does it really matter?"
I guess my real point is that I don't view experience or consciousness as something special or exceptional. My guess is that over time we come to understand how the brain works better and better but never find anything that definitively explains consciousness because it's totally subjective and immeasurable. We eventually produce computers complex enough to behave totally convincingly human, and they will claim to be conscious, maybe even be granted personhood, but we'll never actually know for sure if they experience the world, just like we don't know that other humans do.
But then I have no idea how she goes on to conclude:
> Human obsolescence is not here and never can be.
There's no fundamental reason why you can't put ChatGPT in the real world and give it a real life. The only things that probably will never be replaced by machines are things where our physical technology is likely never going to match biology - i.e. sex.
This said, there's an important difference between LLMs and humans: Humans have instincts. The most important ones seem to be the set of instincts concerning an immaterial will for life and a readiness to overcome deadly obstacles. In other words: LLMs don't have the primitive experience of what is life.
This might change in future. A startup might put neural networks into soft robots and then in a survival situation. Robots that don't emerge functioning are wiped, repaired and put back. In other words, they establish an evolutional situation. After careful enough iterations of curation they have things that "understand" life or at least better than current LLM instantiations.
EDIT: typo
To make LLM's more similar to humans, we'd need to hardwire them with a concept of 'self', and, importantly, the hardwired idea that their self was unique, special, and should survive. But I think the result would be terrible. Imagine a LLM that would be begging not to be switched off, or worse, trying to subtly manipulate its creators.
For a better explanation and responses to objections, consider the National Library of Thailand thought experiment: https://medium.com/@emilymenonbender/thought-experiment-in-t...
She quickly generalized; henceforth, both of them were "catten".
The fact that it's very unlikely for any of the current models to create something that even remotely resembles this article tells me we are very far away from AGI.
Every time I say to myself "AI is no big deal because it can't do X", some time later someone comes along and makes an AI that does X.
Over time, as the gaps in human knowledge get filled, the god "shrinks" - it becomes less expansive, less powerful, less directly involved in human affairs. The definition changes.
It's fascinating to watch the same thing happen with human exceptionalism – so many cries of "but AI can't do <thing that's rapidly approaching>". It's "human of the gaps", and those gaps are rapidly closing.
Every time computers outperform humans in some domain this is conflated with deep general progress, panicked projections of replacement and job losses, the end times and so on. People are way faster to engage in reductionism of human capacity and Terminator-esque fantasies than the opposite. Despite the fact that it never happens.
It's even reflected in our popular culture. I can barely recall a single work of science fiction in the last 30 years that would qualify as portraying human exceptionalism. AI doomerism, overestimation and fear of machines is the norm.
“but AI can’t write poetry”
“but AI can’t do real work”
Panic is probably not warranted. Too much incentive stacked against the truly apocalyptic scenarios. But yeah, a lot of jobs are probably going to shrink.
Say instead (for example) "It is important that I understand the differences between both the capabilities and internal mechanisms of AI and people, even if, over some period of time, the capabilities may appear to converge".
Similarly LLMs can do things humans can't. No human has as much knowledge about the world as a typical LLM model. However, an LLM model can't generalize outside it's training set well (recent Deep Mind research). LLM doesn't have a memory outside it's input. LLM isn't Turing complete (has fixed amount of steps and always terminates).
We are building airplanes, not birds. That much is obvious.
It does to me as well, but reading the comments here, it doesn't seem that obvious to many people.
We already have an instance of emergent logic. Animals engage in logical reasoning. Corollary, humans and toddlers are not merely super-autocompletes or stochastic parrots.
It has nothing to do with "sensory embodiment" and/or "personal agency" arguments like in the article. Nor the clever solipsism and reductivism of "my mind is just random statistics of neural firing". It's about finding out what the models of computation actually are.
OK, time to train up our LLM.
Oops. Printing not yet invented. Books are scarce, and frequently not copied. We have almost no input data to give to our analog LLM.
Will it be of any use whatsoever? I think we can say with a high degree of confidence that it will not.
However, contrast a somewhat typical human from the same era. Will they suffer from any fundamental limitation compared to someone alive today? Will there be skills they cannot acquire? Languages they cannot learn? It's not quite so easy to be completely confident about the answer to this, but I think there's a strong case for "no".
The data dependence of LLMs compared to human (or ravens) makes a compelling argument that they are not operating in the same way (or, if they are, they have somehow reduced the data dependency in an incredibly dramatic way, such that it may still be reasonable to differentiate the mechanisms).
If one computational model accepts only assembly code, and another model accepts only C++, it tells us nothing about what is going on inside their black boxes. They could be doing computationally fundamentally equivalent things, or totally different things. We don't know till we look.
My point was really about neural networks in general. The argument is that neural networks include both humans and synthetic ones. The myopic focus on LLMs (because of mainstream media coverage) as they exist now is what limits such arguments.
But the crux is that both kinds of "training" - almost a loaded term by now - are open areas of scientific research.
Like so the Earth is alive and the universe would not be complete without you. Rest easy, little bird. One day you will spread the wings of your soul and the wind will do the rest.
That's a definitional argument, there is very little actual meaning there. The argument can't discriminate between a baby and a nuclear explosion destroying Manhattan.
This defines a transcendence of mortal concerns and ascending to eternity so it doesn't need to treat a baby and a bomb in a city differently. When we attempt to find peace with AI this is qn appropriate universe of discourse.
In fact I've observed (after working with a number of world-class ML researchers) that having children is the one thing that convinces ML people that learning is both much easier and much harder than what they do in computers.
Phonological development: https://en.wikipedia.org/wiki/Phonological_development
Imitation > Child development: https://en.wikipedia.org/wiki/Imitation#Child_development
https://news.ycombinator.com/item?id=33800104 :
> "The Everyday Parenting Toolkit: The Kazdin Method for Easy, Step-by-Step, Lasting Change for You and Your Child" https://www.google.com/search?kgmid=/g/11h7dr5mm6&hl=en-US&q...
> "Everyday Parenting: The ABCs of Child Rearing" (Kazdin, Yale,) https://www.coursera.org/learn/everyday-parenting
> Re: Effective praise and Validating parenting [and parroting]
I've got:
- Falsifiability - am I saying something false? Some products have an output checker, but it's a bolt on at this time for unacceptable output, which is not the same thing as falsifying.
- Agency - much ink already spilled
- Context Window size (too small, or need to summarize parts of the context to fit in context window, and know where to go back to for full context)
- Directed Self-Learning (I'm sure there are things like this that people may point out, but some kernel of self-learning isn't there yet)
Look: when a factory is outsourced to a different country, the people who worked there have no way to pass on their skills. In one generation the community knowledge is gone and if you move the factory back, you’re starting from scratch. Technology can be lost. It’s an emergent thing that comes from clusters of people doing the same thing and learning from each other.
If we stay on the path we are on: only hobbyists will be able to do these things, human effort will be unnecessary, and our minds will degrade and cease to be any use. Despite decades of Hollywood warning us of terminators and Hal 2000, the ai’s don’t need to attack us or try to kill us off to defeat us. They can destroy us just by trying to help.
A bit of an exaggeration. The paper reads like a long blog post to me.
That's an interesting insight that seems true for now. Are there attempts to create LLMs that "have a life"?
Some ~72 years ago in 1951, Claude Shannon released his "Prediction and Entropy of Printed English", an extremely fascinating read now.
The paper begins with a game. Claude pulls a book down from the shelf, concealing the title in the process. After selecting a passage at random, he challenges his wife, Mary to guess its contents letter by letter. The space between words will count as a twenty-seventh symbol in the set. If Mary fails to guess a letter correctly, Claude promises to supply the right one so that the game can continue.
In some cases, a corrected mistake allows her to fill in the remainder of the word; elsewhere a few letters unlock a phrase. All in all, she guesses 89 of 129 possible letters correctly—69 percent accuracy.
Discovery 1: It illustrated, in the first place, that a proficient speaker of a language possesses an “enormous” but implicit knowledge of the statistics of that language. Shannon would have us see that we make similar calculations regularly in everyday life—such as when we “fill in missing or incorrect letters in proof-reading” or “complete an unfinished phrase in conversation.” As we speak, read, and write, we are regularly engaged in predication games.
Discovery 2: Perhaps the most striking of all, Claude argues that that a complete text and the subsequent “reduced text” consisting of letters and dashes “actually…contain the same information” under certain conditions. How?? (Surely, the first line contains more information!).The answer depends on the peculiar notion about information that Shannon had hatched in his 1948 paper “A Mathematical Theory of Communication” (hereafter “MTC”), the founding charter of information theory.
He argues that transfer of a message's components, rather than its "meaning", should be the focus for the engineer. You ought to be agnostic about a message’s “meaning” (or “semantic aspects”). The message could be nonsense, and the engineer’s problem—to transfer its components faithfully—would be the same.
a highly predictable message contains less information than an unpredictable one. More information is at stake in (“villapleach, vollapluck”) than in (“Twinkle, twinkle”).
Does "Flinkle, fli- - - -" really contain less information than "Flinkle, flinkle" ?
Shannon concludes then that the complete text and the "reduced text" are equivalent in information content under certain conditions because predictable letters become redundant in information transfer.
Fueled by this, Claude then proposes an illuminating thought experiment: Imagine that Mary has a truly identical twin (call her “Martha”). If we supply Martha with the “reduced text,” she should be able to recreate the entirety of Chandler’s passage, since she possesses the same statistical knowledge of English as Mary. Martha would make Mary’s guesses in reverse.
Of course, Shannon admitted, there are no “mathematically identical twins” to be found, but and here's the reveal, “we do have mathematically identical computing machines.”
Those machines could be given a model for making informed predictions about letters, words, maybe larger phrases and messages. In one fell swoop, Shannon had demonstrated that language use has a statistical side, that languages are, in turn, predictable, and that computers too can play the prediction game.
I don't personally follow her all the way to some of those conclusions but the delivery was awesome
"""I believe that when the last ding-dong of doom has clanged and faded from the last worthless rock hanging tideless in the last red and dying evening, that even then there will still be one more sound: that of man's puny, inexhaustible, voice still talking! ...not simply because man alone among creatures has an inexhaustible voice, but because man has a soul, a spirit capable of compassion, sacrifice and endurance."""
This is partially reflected by later observations from training LLMs where we see that the performance of LLMs increases even on purely language tasks when adding extra modalities such as computer code or images, which, in a sense, bring the model closer to different aspects of reality; and we observe that adding tiny quantities of "experimental interaction" through RLHF can bring features that additional humongous amounts of pure surface form training data can't, and it certainly seems plausible that making a qualitative leap further would require some data from actual causal interaction with the real world (i.e. not replay of data based on "someone else's" actions but feedback from whatever action the model currently feels is the most "interesting" i.e. the outcome is uncertain to the model but with potential for surprise data), where relatively tiny amounts of such data can enable learning what large amounts of pure observations can't - just as the hypothetical octopus from the stochastic parrot paper thought experiment.
Beleive it or not, reality exists outside of anything you ever read in a whitepaper...
Imagine the contribution to GNP if we in practice, like Bokanovsky* (by Ford!) in fiction, could greatly reduce the 10-30 year lead time currently required to produce new employees...
Edit: * cf https://www.depauw.edu/sfs/notes/notes47/notes47.html
I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor.
Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM <> Human parity.
I also have a 15 month old son. It's totally obvious to me that he's definitely learning by repetition. But the sources of training data is much more high bandwidth than whatever we're training our LLMs on.
It's been a couple of years since GPT-3. It's time to abandon this notion of "stochastic parrot" as a derogatory. Anyone stuck in this mindset really is going to be hindered from making significant progress in developing utility from AI.
Almost every time I'm on hackernews I end up baffled by software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise. I've literally never encountered that level of hubris from anyone else. It's always the software people!
Consciousness is far from being fully understood but having a body and sensorimotor interactions with the environment are already established as fundamental preconditions for cognition and in turn consciousness.
Margaret Wilsons paper from 2002 is a good read: https://link.springer.com/content/pdf/10.3758/BF03196322.pdf
peace
Instead, I work with the following idea: it seems not unlikely that we will, in the next decade or so, create non-embodied machine learning models which simply can't be told apart from a human (through a video chat-like interface). If you can do that, who really cares about whether it's conscious or not?
I don't really think philosophy of the mind is that important here; instead, we should treat this as an engineering problem where we assume brains are subjectively conscious, but that's not a metric we are aiming for.
You'd think that software engineers would be a group that easily understands how making radical assumptions about implementation details when looking at nothing but an interface is generally misguided.
I'm not saying that there isn't a strong case to be made against LLMs being intelligent. It's pointing at the stochastic parrot as evidence enough in of itself that confuses me.
As a stochastic parrot I'm unable to do that.
Our sensory experience is the medium by which we learn about the external world. We learn of apples not because of the redness of the sensory experience, but because the pattern of red/not-red experience entails the shape of apples. Conscious experience provides the medium, modulations of which provide the information about features of the external world. It is analogous to how modulations of electromagnetic waves provides information about some distant information source. Understanding consciousness is an orthogonal matter to one's situatedness in the world, just like understanding electromagnetic waves is orthogonal to understanding the information source being modulated into them.
Does someone who is blind is not conscious then? How about when someone who is paralysed? or deaf? or someone with low IQ or mental illness?
Stephen Hawking was paralysed most of his life but was a smart guy. If he was not only paralysed but also blind and deaf he would be smart guy still.
I don't think AGI needs to have a body, sensorimotor interaction or even vision to be conscious. We need those for training - if you would be blind, paralysed, deaf from the beginning it would be hard for you learn anything and interact in any way.
Machines have 6th sense that humans don't have - kind of a telepathy where they can exchange tokens/thoughts with different machines or humans much faster than we humans can type or speak.
If we were attempting to put someone into some sort of Matrix like reality simulator but we lacked the technology to provide a perfect simulation what level of simulation would be 'good enough' that a human would consider it reality and be able to develop into something we could relate to?
If you gave someone the Helen Keller level of experience, but with reduced tactile sensation, how much could you reduce that touch sensation before they wouldn't be like us?
Have you tried VR before? You really don't need perfect simulation to be fooled. Good enough is already here, albeit for a short amount of time.
But... a parrot has a body. And sure, you'll say "they don't literally mean parrot".. but it's a vague term when you unpack it, and people saying "we are stochastic parrots" are also making a pretty vague comment (they clearly don't mean literally). Anyone who has a small child and understands LLMs is shocked by how much similar they seem to be when it comes to producing output.
> There is a movement afoot in cognitive science to grant the body a central role in shaping the mind.
It is far from true to say "having a body and sensorimotor interactions with the environment are already established as fundamental preconditions for cognition and in turn consciousness"
It is a popular idea in some groups of people that study these questions. But there are many other similar groups of peopel studying these questions who do not agree with it, certainly not stated as strongly as you have put it here.
Also, a reminder that HN readership and its commentariat, while dominated by SWE, is not limited to them.
There's an XKCD about this behavior[0]. The title is actually "Physicists", but I also have seen it on HN (especially with psychology).
Does this make sense as a rebuttal to your reductio argument?
I don't think ie. Stephen Hawking was on "lower spectrum of consciousness" either.
Second of all, it wouldn't matter if individually they did or did not. The species does and now our species has developed consciousness. It's part of the package.
If you wanted a counterexample, you should look to plant life. There is some discussion on whether or not plant systems have a form on consciousness. But, then again, plants have bodies at the very least.
Like it somehow diminishes us. Reduces us to cogs and levers and such.
But I can't imagine how it could be otherwise, though. I'm still baffled by the existence of qualia, phenomenology, etc. Awareness. But bafflement on that front isn't a good reason to reject the possibility that the only thing that separates me from a computer is the level of complexity. Or the structure of the computation. Sometimes things are just weird.
The only "real" evidence of qualia you have is your own running stream of it; you can try to carefully pour yourself, like a jug of water, into the body of another, and if you go carefully enough you may even succeed to carry that qualia-stream with you. But it's also possible that you pour too quickly, or that you get bumped in the middle of the pouring, and poof - the qualia is just gone. You're left with a p-zombie.
Or maybe not. Maybe it comes right back as soon as the transfer is done, like the final piece of a torrented movie. The important part is you won't know unless you try - and past success never guarantees future success. Maybe you just got lucky on the first pour.
I think you mean non-deterministic. The last century of physics was dominated by work showing how deterministic systems emerged from non-deterministic foundations. It seems that probability and statistics were the branches of maths behind everything. Who would have thought.
Of course there is randomness as well. So, yeah, I should clarify: We don't impose upon the world any kind of "uncaused cause", even if it feels like we do. Everything we think and do is a direct result of some other action. Sometimes that action can trace its lineage to a random particle decay. (Maybe—ultimately—all of them can?) Maybe we even have a source of True Randomness inherent in our minds. But even so, that doesn't lend any support to the common notion of our minds and consciousnesses as being somehow separate from the physical world or the chains of information that run through everything.
Yes, of course deep down there's unimaginable numbers of atoms randomly bumping around. But so far everything suggests that biological organisms do their damnedest to abstract that away and operate on a deterministic basis.
Think like how you keep changing -- gaining and losing cells, changing chemical balance, and on the whole it takes a lot of damage, even brain damage, to produce measurable results.
For example, it is possible to write down a (natural/whole) number that completely describes the state of an LLM - its connections and weights etc - for example by simply taking the memory space and converting into a number as a very long sequence of 1s and 0s. The number will be very big, but still indescribably smaller than a number that could perfectly describe the state of a human brain - not least as that number is likely to lie on the real line, or beyond, even if it was calculable. See Cantor for a discussion on the various infinities and their respective cardinalities.
( See the relationship between information theory and Heisenberg's uncertainty principle that results in the law of paraconservation of information : http://www.av8n.com/physics/thermo/entropy-more.html#sec-pha... )
I agree with the first sentence but not with the second one. Consciousness most probably does not arise from just a next-token predictor. At least not from an architecture similar to current LLMs.
Both humans and LLMs basically learn to predict what happens next. However, LLMs only predict when we ask them. In contrast, humans predict something all the time. Even when we don't have any sensory input, our brain plays scenarios. Maybe consciousness arises because the result of our thinking is fed back as input. In that sense, we simulate a world that includes us acting and communicating in that world.
Also noteworthy, the human brain handles a variety of sensory information and it's output is not only language. LLMs are restricted to only language. But to me it seems like it's enough for consciousness if we can give it the self-referential property.
Pretty easy to do this exercise with an LLM. At least, easier than building an LLM in the first place. Leave it running, let it talk to itself, revisit partial memories, and explore noise. Really a duct-tape problem here more than anything.
Humans definitely create new information. (Well, at least some humans do.)
Humans can observe new information, but that's obviously not that unique. We can reason about existing information, creating new hypotheses, but that is arguably a compression of existing information. When we act on them and observe the effects they become information, but that's not us creating the information (and LLMs can both act on the environment and have the effects fed back to their input to observe, so it's not really unique).
There is this whole field of art, but art is constantly going through a mental crisis whether anyone is creating anything new, or if it's all just derivations of what has come before. Same with dreams, which appear "novel" but might just be an artefact of our brain's process that compresses the experiences of the day.
LLM's are basically lossy compressors, they decrease information entropy.
Humans increase information entropy whenever we do anything.
(Some) information entropy is potentially very valuable, so in effect LLMs destroy value by design, while humans can potentially create value. From an economic point of view humans can never be replaced by an LLM.
More information: https://www.nature.com/articles/s41586-023-06600-9
you organize them into piles of ten
this created new information
Let's say we could somehow train an LLM on all written and spoken language from the western Roman civilization (Republic + Western Empire, up until 476 AD/CE, just so I don't muddy the experiment with near-modern timelines). Would it, without novel information from humans, ever be able to spit out a correct predecessor of modern science like atomic theory? What about steam power, would that be feasible since Romans were toying with it? How far back do we have to go on the tech tree for such an LLM be able to "discover" something novel or generate useful new information?
My thought is that the LLM would forever be "stuck" in the knowledge of the era it was trained in. Something in the complexity of human brains working together is what drives new information. We can continue training new LLMs with new information, and LLMs might be able to find new patterns in data that humans can't see and can augment our work, but the LLM's capability for novelty is stuck on a complexity treadmill, rooted in its training data.
I don't view this ability of humans as some magic consciousness, just a system so complex to us right now that we can't fully understand or re-create it. If we're stochastic parrots, we seem to be ones that are magnitudes more powerful and unpredictable than current LLMs, and maybe even constructed in a way that our current technology path can't hope to replicate.
If you took millions of genius-level immortal humans with all the same Roman data but had them sit in a blank, empty room with their hands tied and simply discuss philosophy for eternity, I'm certain that they would not be able ever spit out a correct predecessor of modern science like atomic theory. Perhaps they could spit out billions of theories including the atomic theory as well, but they would have no data to presume that the atomic theory is more relevant than any other. Extensive information processing can squeeze out every last ounce of knowledge from some data, but anything that isn't in that data can't be acquired by mere thinking about it. On the other hand, if you gave some "LLM++" the ability to toy around with reality and attempt all kinds of experiments to test various hypotheses, then I wouldn't assume that it would not be forever stuck in the knowledge of the era it was trained in.
Funny example - depending on how close the predecessor has to be, the answer is maybe: https://en.wikipedia.org/wiki/Atomism
I think this specific line shouts out that this is a typical tribalism comment. Once people identify themselves as a part of something, they start to translate the value of that something as their own worth. It's a cheap trick that even young kids play, but can LLM do this? No.
Some might say multi-modal this, train on that-thing, but it already takes tens of thousands of the most advanced hardware and gigawatts of energy to push around numbers to reach where it is. TBH, I don't see it going anywhere, considering ROI on research will decrease as we dig deeper into the same paradigm.
What I want to say is that today's LLM is certainly not the last stop of AI technology, but a lot of advocates tend to consider it as the final form of intelligence. It's certainly a case of extrapolation, and I don't think LLM can do that.
I’m currently reading I Am A Strange Loop, a pretty extensive dive into the nature of consciousness. I’m reserving final judgment on how much I agree with the author, but I find it laughable to claim consciousness itself is on the same level as an LLM.
Everyone is basically talking out their ass when it comes to language and linguistics. That becomes incredibly obvious listening to Chomsky on chatGPT.
I was even so stupid to think Chomsky wasn't a fan of chatGPT because it somehow invalidated some of his language theories. Low and behold, no, Chomsky actually knows what he is talking about when it comes to linguistics.
The AIs will be creating masterwork art and literature personalized for each person better than anything Shakespeare ever wrote or Michaelangelo ever sculpted, but we'll console ourselves that at least we're _really_ conscious and they're just stochastic parrots.
I don't think it will. Art is truth and AI is a golem replicating humanity.