Yann LeCun on GPT-3
facebook.com
facebook.com
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it. It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians.
But compiling massive amounts of operational knowledge from text is still very much a research topic.
> GPT-3 doesn't have any knowledge of how the world actually works.
I think this is a philosophical question. There is a view that, basically, there is no such thing as knowledge, just language (or, at least, there is no distinction between knowledge and language). In this view, all there really is is language, which is mostly composed of metaphors and, ultimately, metaphors only refer to other metaphors, i.e. language is circular. In this view, not only is the ultimate, physical, concrete world beyond us but also we can't even talk about it. From this perspective, GPT-3 is not substantively different than what our minds are doing.
That view makes some strong claims (I don't find it convincing), but it's out there. A slightly different claim, though, is that "knowledge of how (we think) the world actually works" is encoded in language. To me, that seems trivially true. So, again, how you take this quote from LeCun depends on what you think knowledge is and your view of the relationship between knowledge and language.
Logical empiricism was ultimately a dead end as the criteria for even verifying empirical truth has long been contentious philosophically, and was further critiqued by contemporaries such as Quine who attacked the premise of the analytic/synthetic distinction (think Hume's fork, which Kant tried to solve) and Popper who cited the problem of induction to critique the fundamental premises of the positivists verificationism.
Wittgenstein is an interesting case, as the Tractatus is considered an early work of his, profoundly influential to analytic philosphy at the time, yet his later work, Philosophical Investigations is sometimes seen to retract the dogmatism found in the Tractatus. I tend to take the view that it's a continuation of his thought, rather than a retraction of his earlier work. Crudely, whereas his former thought represented a narrowly axiomatic definition of language and its truth value, PI investigates, among many other ideas, language as an activity, or game, that has meaning dependent on the context of its use, languages as families. Granted, Wittgenstein is a complex thinker and these are simply my interpretations.
It's also curious to note that as positivism was beginning to fall out of favor around the time of the second world war, a continental thinker such as Heidegger, whose thought luxuriated in the kind of metaphysical questions the positivists necessarily eschewed, rose to prominence and was infamously sanctioned by the NSDAP to philosophize about their presumed "destiny". Bit of a tangent, but I think the historical context is relevant, as often philosophical movements are birthed from pre- and post-war attitudes.
Of course, you could always attempt to define knowledge such that it is purely verbal, or alternatively define whatever is going on in the brain of an animal to be language, but is either approach useful? In common usage, we recognise, as knowledge, various things that cannot be communicated by language, such as knowing how to ride a unicycle on a tightrope (I doubt you can learn it just from a book) and the infamous qualia which supposedly prove that the mind is dualistic. And what about the knowledge of how to use language? How does that get bootstrapped?
This can lead to some confusion. In Frank Jackson's Knowledge Argument[1], dualists say that qualia are knowledge, knowledge is language, and so if Mary could not learn qualia by studying science, then materialism is false. This argument trades on being inconsistent over what it means to have knowledge of something.
In our heads, language is a combination of words and concepts, and knowledge can be encoded by making connections between concepts, not simply words. If there is no concept or idea backing up the words, it can hardly be called knowledge. Consider the case of the man who did not speak French, yet memorised a French dictionary, and subsequently went on to win a Scrabble competition. Just because he knows the words, would you say he knows the language?
A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
Great point.
> A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
Are you sure? Aren't "concepts" encoded in how language is used, at least to some degree?
LeCun does say that models that explicitly attempt represent knowledge perform better than GPT-3 in terms of answering questions. I'm no expert but I believe him.
Good point and I think this shows up to the extent different languages might affect how we express particular concepts.
However I think it is more accurate to say that language solidifies and gives form to how we express concepts and the “concepts” themselves are independent of languages. Only our “expression” of these “concepts” depends on language.
For anyone interested in art and art history, this distinction was the central focus of the French surrealist painter Rene Magritte.
> We’re very far from having machines that can learn the most basic things about the world in the way humans and animals can do. Like, yes, in particular areas machines have superhuman performance, but in terms of general intelligence we’re not even close to a rat. This makes a lot of questions people are asking themselves premature. . That’s not to say we shouldn’t think about them, but there’s no danger in the immediate or even medium term. There are real dangers in the department of AI, real risks, but they’re not Terminator scenarios.
That's pretty measured overall, but he doesn't know that there's no existential AI risk in the medium term. No one does, and that's the problem. Experts simply suspect that it's unlikely. Stuart Russell and him have debated similar topics [2].
To tie back to your point: I keep seeing LeCun brush over tricky questions like yours and the ones at [2] with an arrogant confidence. I wish that he would be more careful, and I hope that I have a skewed view of him.
[1] https://www.theverge.com/2017/10/26/16552056/a-intelligence-...
[2] https://www.lesswrong.com/posts/WxW6Gc6f2z3mzmqKs/debate-on-...
Except people are bad at exponential processes. Yet when economics drives us we are suddenly good at making them happen. And this combo seems to be what makes these existential risks. (Like climate change, or other manifestations of the coordination problem.)
Most expressions of language that survived from a few thousand years ago are centered around myths, and while those myths may have contained certain moral or ethical lessons (that were and are subject to interpretation) they certainly weren't tightly coupled to reality in an objective sense.
Training on expressions of language (I separate the concept of language itself from its expression in the form of writing, speaking, etc) certainly has use cases but can GPT-3 recognize a previously unknown analogy and correlate it with the proper piece of applicable "knowledge" it has? If not then it really has no understanding.
1. i get where you're coming from
2. yes, language is bottlenecked by human perception, as are all things
3. even the notion of myth and fiction is encoded in language. language is self-descriptive and self-aware and you can separate sense from non-sense.
4. i'm not talking about knowledge or understanding, but of addressing the question of why training on language let's GPT-3 make human-like predictions as if it knows about reality? either it's a fluke, or it's because language as a whole is a model that approximates reality.
There's little doubt that GPT-3 could hold its own quite well in a bout of polite conversation and/or small talk that features in many social situations but that's more a commentary on the limited area of knowledge and behavior that etiquette expects for interactions in such settings.
It could also be trained on the canon of Shakespeare and behave as a prior work of Shakespeare, but if you left one play out of that canon and then attempted to have GPT-3 generate that missing work it wouldn't... at all. It doesn't approximate how Shakespeare thought, it approximates the literature he produced that you used for training.
None of this is to denigrate the achievement that GPT-3 represents, it is merely to point out that it is unfair to GPT-3 to attempt to hold it to the standards of AGI.
Do you really think that humans are so special as to encode all their knowledge in language? Watch a movie. Listen to a song. Examine a piece of art. Feel sculpture. Play a guitar. Dance.
There is a segment of the software community that is highly language centric/adept. But that community is often blind to other forms of understanding.
Just look at the language of Shakespeare. Much of the language is visual and experiential. How much would you actually understand without your senses and imagination? Your knowledge encompasses your being.
Well, actually, yes, they do. Many animals have elaborate languages encompassing many concepts. Crows can explain to one another what a human looks like, for example.
The real mechanistic view drops language as a special case and just says their is no knowledge, only behavior.
If there were two versions of this poem that started the same way, it would pick between the variations in the corpus randomly. In other cases it might choose based on the style of prose or other stuff like that.
GPT-3 can get some trivia right, but it's only because the editors of Wikipedia already came to consensus about it and Wikipedia was weighted more. It doesn't have a way of coming to a consistent conclusion on its own.
Without consistency, how can it be said to know or believe anything? You might as well ask what a library believes. Sure, the authors may have believed things, but it depends which book you happen to pick up.
But I do think you're missing the point just a bit. When we speak and think, we use all kinds of metaphors that express judgements about the world, usually without realizing it. In other words, the way we use language encodes concepts in a deep way.
To borrow an example from George Lakoff, we, in English, use war-metaphors to talk about arguments. Of arguments and of wars you can say things like "he's marshalling his forces," "they're ceding their territory," or "she's girding her defenses". In fact, almost anything you can say about a war you can also say about an argument. In American politics, with regard to partisan squabbling and the filibuster, we talk about "the nuclear option". The fact that these metaphors make sense to us indicates a judgement, something like "arguments are like wars". That judgement shows up in billions of lines of English scraped from the internet and can be fed into a model, allowing GPT-3 to "make that connection" via purely statistical methods.
Yes, this is a bit like asking "what a library believes". But a lot of these metaphors show up in our languages and, in a way, they express judgements, which is something akin to a belief. Does that mean a library has beliefs? Is this all knowledge is? I wouldn't go that far. But the argument is an interesting one and worth raising.
But a sophisticated understanding of metaphors could be used to tell the truth or to lie. In the case of GPT-3, it doesn't know the difference. Telling the truth and lying come out of the same autocompletion process.
If you consider the use of a metaphor to be showing judgement, it means that a particular metaphor seems to be appropriate to use in a particular context.
If it's only philosophical, then me saying that Hacker News Website itself has 'knowledge' of everything we discuss about is also philosophical. Same can be applied to plain paper books.
How about any web application? A for loop? anything which can generate something for you?
But in my experience, the few-shot learner attribute of GPT-3 makes it insanely useful. We have already found several use cases for it, one of which replaces 2 ML engineers.
Yes, it's not perfect, but it's pretty good at many things, and REALLY easy to use.
Also, if you've found a use case where GPT-3 replaces real humans, what did those humans actually spend their time on? Seems like either you're over-hyping GPT-3, or under-hyping humanity
You just change the prelude and use it for something new.
But you have to understand, the use cases you mention are shallow and limited. The heart of GPT, the fine-tuning, is gone. And it looks like even OpenAI gave up on letting users fine-tune, because it means they essentially do build an entirely new, expensive model for each use case.
I wanted to make an HN Simulator, the way that https://www.reddit.com/r/SubSimulatorGPT2/ works. But that's far beyond the capabilities of metalearning (the idea that you describe).
However, even if they are, the point stands: currently, there are teams of people at companies all over the world tuning models for these shallow and limited use-cases. GPT-3 can replace them all, without OpenAI needing to invest another cent in training for a particular customer's use-case. That is in fact game-changing for the ML/DL world and current applications thereof.
Is it AGI? Obviously not. But the vast majority of ML applications don't need to be.
(https://www.reddit.com/r/SubSimulatorGPT2/ but for HN.)
For a more extensive rebuttal, I wrote one here. https://news.ycombinator.com/item?id=23346972 Though that was more a rebut of GPT in general as a path to AGI than metalearning in particular for generating memes.
That being said, I'm not sure I understand why you can't use GPT-3 to make an HN simulator.
The counterpoint is that it would be significantly cheaper AND have better performance to fine-tune models to each customer's use case than it is to just run GPT-3 at inference.
edit: GP giving more downvotes than proofs
GPT-3 is not doing what the ML engineers were doing (building models), GPT-3 is the end goal. The company just decided to outsource the work to OpenAI and pay a monthly fee to them instead of salaries to their ML engineers.
"We have already found several use cases for it, one of which replaces 2 ML engineers." -> Clearly makes it sounds like GPT-3 can do the job their ML engineers were doing.
I think everyone understood that.
One thing GPT-3 is not able to do for example, is replacing 2 ML engineers to build a GPT-3 like model. But OpenAI can do that.
Mostly sarcasm, mostly.
IDEs and higher level tools help engineers become more productive. They can do more, with less. This raises the bar on products, and the demand of customers for them (things are pretier, easier to use, etc..), which in turn creates more domains for software to be used, and more demand for engineers.
Google "Induced Demand"
If you think this isn’t a concern, I’ve already seen it happen with my own eyes, rather than hearing about it second hand. They encouraged someone to make a writing tool. That someone then spent roughly six weeks prototyping, iterating, and giving constant feedback. All signals from OpenAI were “Yes, awesome!”
Then one day they simply declined to let them ship. Anything. Anything even resembling “a tool to generate huge quantities of outputs.” Which was, you know, the whole point.
You play, you pay. And I hope you’re ready to pay, because you won’t have your magical genie unless the magical genie’s caretakers believe you are sufficiently worthy.
I cloned as much of OpenAI’s API as I could: https://twitter.com/theshawwn/status/1312299759592333318?s=2...
All that’s left is to reproduce a substantially similar model. Which is doable, but will take time. In the meantime, tread carefully.
But you can't pay employees with cloud credits, so time will tell whether it was a correct decision. (It probably was. And I exaggerate slightly; the investment included a substantial sum of real dollars too. But most people see that billion dollar investment and think it's all dollars, when in fact it was largely credits.)
That said, I've spent a lot of time with it this month and think it will be an extremely useful tool for creative works of all types. It's not to a point where you can just tell it to write a blog post (yet!) but it can generate novel snippets, ideas, and variations that are actually usable. Unskilled creatives should be worried. Skilled creatives should incorporate it into their workflow.
Just want to point out that he's saying the people on the upper end of the expectation distribution are wrong, not the people in the middle of it. So if you're takeaway from this is that GPT3 is nothing special, that's probably the wrong message.
That's not just "some people have unrealistic expectations" it's "this tool, when when more advanced and find tuned, will never be appropriate to use in a very broad class of use cases".
He also says "GPT-3 has no knowledge of how the world works", which is clearly an overstatement meant to clear up hype, but is untrue. For example, GPT-3 knows more trivia than I do.
It all depends on your definition of knowledge. Under a certain definition you could say that GPT-3 knows basically nothing.
If someone teaches me to repeat perfectly something very smart in a language I don't know, without explaining to me what that thing is, do I have knowledge about this?
The same argument can be made about those kind of models, the knowledge they have is about the structure of the language and what word is most likely to come next, but they have no way to ground those words in actual relation with the world.
Yes that's what leads to multiple definition of what knowledge really is. Yann LeCun believe we are more than just that, hence why he is saying GPT-3 would have no knowledge.
no it doesn't, GPT-3 is a very sophisticated parrot. it doesn't know any trivia, it knows how to put the most likely string of characters next to the one it just saw, it doesn't matter what the text represents. That's the difference between you and the model.
It's basically the Chinese room. You can make an analog GPT-3 by asking a question, recording your answer, handing someone who doesn't understand a word of your language the giant box of tapes, and she tries to match them together until she appears to make sense to listeners
It just also "happens" to be the case that if you interpret those pixels as go moves and play those moves against the world champion human go player then eventually that human will hold a press conference announcing to the world that AlphaGo has won the match.
Deep Mind made a 'superhuman' Breakout bot, but if you messed with the panel at the bottom it sucked. It hadn't learned what the panel is or what the game is about, it had just learned to rearrange pixels. You can get away with this in one version of Breakout, but in the real world it's brittle and useless
"Q: Who was president of the United States in 1801? A: Thomas Jefferson was president of the United States in 1801.
Q: What city is in the northwest corner of Ohio? A: Toledo is in the northwest corner of Ohio.
Q: What highway do you take to get to North Lake Tahoe from Oakland? A: Highway 80 is the highway you take to get to North Lake Tahoe from Oakland.
Q: What are Darius's abilities in League of Legends? A: Darius's abilities in League of Legends are: Apprehend, Decimate, Noxian Guillotine, Crippling Strike, Apprehend [sic], and Hemorrhage.
Q: Who did Capablanca defeat to become world chess champion? A: Capablanca defeated Lasker to become world chess champion."
Of those 5 questions, I could answer #1 after deploying a mnemonic and some math, and #2 if you gave me multiple choice of the top 5 cities in Ohio, and I would miss the rest.
Edit: in Ireland, on Firefox desktop
Facebook seems to put different restrictions depending on where you live. I'm based in Western Europe and never been able to read anything from Facebook without logging in. Same for Instagram.
EDIT: Actually - that's no excuse for that awful second sentence. I'm ashamed of myself.
Personally I see little evidence that this "just scale a transformer until sentience" hype-train is going to take us anywhere interesting or particularly useful.
And for the people who claim it is super useful already, can you actually trust its outputs without any manual inspection in a production setting? If not it's probably not as useful as you think it might be.
GPT-3 on Progress. “Civilization rose on the exponential curve. We shouldn’t expect progress to follow a straight line.”
Google with date restriction before 1 April 2020: "progress to follow a straight line".
Do you see only one result that come :-)
https://chrismukiibi.com/2019/11/26/the-valley-of-disappoint...
"We shouldn’t expect progress to follow a straight line." and "we expect our progress to follow a straight line"
Do you understand now, how they use conjunctions(joining words) to insert or to delete insignificant words as "shouldn't" and "our" to plagiarize so that they are not caught.
"After two days of intense debate" "the United Methodist Church has agreed to" "one that is expected to end" "in the creation of a new denomination"
You will find and sources of GPT-3 text:
After two days of intense debate, the United Methodist Church has agreed to a historic split – one that is expected to end in the creation of a new denomination, one that will be “theologically and socially conservative,” according to The Washington Post. The majority of delegates attending the church’s annual General Conference in May voted to strengthen a ban on the ordination of LGBTQ clergy and to write new rules that will “discipline” clergy who officiate at same-sex weddings. But those who opposed these measures have a new plan: They say they will form a separate denomination by 2020, calling their church the Christian Methodist denomination. The Post notes that the denomination, which claims 12.5 million members, was in the early 20th century the “largest Protestant denomination in the U.S.,” but that it has been shrinking in recent decades. The new split will be the second in the church’s history. The first occurred in 1968, when roughly 10 percent of the denomination left to form the Evangelical United Brethren Church. The Post notes that the proposed split “comes at a critical time for the church, which has been losing members for years,” which has been “pushed toward the brink of a schism over the role of LGBTQ people in the church.” Gay marriage is not the only issue that has divided the church. In 2016, the denomination was split over ordination of transgender clergy, with the North Pacific regional conference voting to ban them from serving as clergy, and the South Pacific regional conference voting to allow them.
So Yann LeCun is not right about GPT-3 here. They don't try to predict anything - especially "one word at time". They simply plagiarize big clusters from 5-7 words or whole sentence and do cosmetic modifications to them to be not recognized by direct searching with Google. When you plagiarize such big clusters written by humans it is normal output compile result of GPT-3 text to have human like look for readers. And they don't use statistic for nothing else except the plagiarism process itself. It will be strange ML algorithm to use phrases with low statistical value as "in the creation of a new denomination" or "progress to follow a straight line". ;-) When you can't emulate human brain processes you can always simulate human plagiarize traits. Especially when there are so many idiots who believe on closed open source code and don't ask elementary questions as "Why someone project suddenly will start to show such "astonishing" results exactly when he close the code behind it?"
Or how they still owe me a bug bounty reward from 2017 for bug in their login form that permit to reap their whole users phones database - and I as last fool give it to them in official channels without to exploit it. For "tanks" they hack my pc and steal my intellectual property you see above.
Next year 2018 2 weaks before big hack I warned them that I founded "new" bug but this time they first must pay me bug bounty reward for this in 2017, and next I will give them new one. THEY Refuse to pay me AGAIN. The rest is history...
I waited FBI to arest me - they don't want to do i even I never hide my identity. Because they must confess they hack my pc, they owe me a money AND THEY DON'T CARE ABOUTSECURITY OF THEIR USERS. I very like how I faced them :---
https://www.facebook.com/login/identify/?ctx=recover&ars=fac...
Then I asked myself a simple question "What if...?" Facebook has around 2 billion monthly active users in 2017. This was roughly about 1/4 of the human population. "What if...I make a BOT to create RANDOM phone numbers and it starts making requests to Facebook's servers to reap their user phones, profile photos and names from their database?" Their users are so many that I will always hit a number sooner or later if there is no protection system against automated requests.
Guess what? I start to reap phone number after phone number of their users and Facebook had no proper protection against automated bots requests :-)
Carefully considered synchronized bots attack on Facebook servers and soon you will have the phones, picture and names at least of 1/6 of the earth population. How much will this information cost?
But I was moral enough (and dumb) to provide them with information about this SECURITY bug on the proper channel and Facebook technical support. 'That would at least bring me a few thousand dollars' I thought naively then. After all, information about this bug was sent to them according to all their rules for "Bug Bounty", I have not retrieved information about their users except to verify this security breach... I even sent them Python program code with which they can see for themselves how it retrieve phones from their database.
Not only did they not pay me a penny for bounty reward - they didn't tell anyone about this security breach in 2017 - they quietly fixed their "login form" so that this bug can no longer be used and exploit.
This happens when you try to be moral and there are crooks in front of you.
$1M, for the service I did for you in 2017. I understand why you hacked my personal computer and your desire to check if I had extracted personal information about your users BUT to steal intellectual property from it that does not belong to you... https://www.sciencemag.org/news/2017/11/artificial-intellige... ... this is the reason why you will pay me $1M not $10K as for ordinary bug bounty reward.
It will be very strange when someone Emil Enchev from Bulgaria publishes an article in https://arxiv.org/ which demonstrates translation algorithm which not need a two-way dictionary, something you're just talking about yet, and he claims that you plagiarized from him by hacking into his personal computer.
It can easily be shown that you have changed your login form from 2017 exactly to block exploiting of bug I talk about - and the question will be how do I know why you made this change when you have never announced this publicly?
It will be a little awkward if you try to challenge the new rising star in AI which demonstrates new algorithms in this field (far above what Facebook owns) that have no analogue, carefully described and with all the program code behind them.
But the catastrophe for Facebook will come after that. What will happen when I open my mouth about 2018 "BIG FACEBOOK HACK" and SEC? So, Mark, I know you understand perfectly well what I'm talking about. Better someone from Facebook to contact me on the specified email and we to solve the problem with your financial obligations to me. I told you above $1M now, or $100M after several months - your choice.
emilenchev1978@yahoo.com
Our copyright dispute will end quickly Facebook.
P.s. And to tell you the truth, in addition to the scandal that will break around Facebook and you personally, I doubt whether you will actually be able to bribe the SEC and stop the subsequent investigation. But as always you know better. You're trying to fuck the devil - ask Boeing and Dennis Muilenburg what was the result of their same experience. Better pay Idiots. I really have no great desire for my name to be involved in such a scandal, from the very beginning of its creation and promotion around the world. But I can't afford to have some morons,like you, to talk after time how they fuck me in the past, while drinking in bars ;-) No one will fuck me with impunity. I hope you understand what I mean, even with my broken English.
GPT is not trying to make a point and is not capable of changing its mind. You, hopefully, are.
Edit: I don't think you should be getting downvoted because it's a valid (and interesting) question.
But this uncovers difficult questions about free will. If we're all just autocompleting based on a combination of the world around us, our internal state, and the physical laws, then what even is intelligence anyway? This view reduces thought to nothing more than an interesting dust storm.
Still, I find the original argument compelling, if not logically convincing. There does seem to be something missing from GPT-3 that differs fundamentally from human intelligence or AGI. But maybe that's an illusion.
Edit: I don't think you should have been downvoted, since your question is valid and constructive in my view.
That's pretty much it. I do belive it's possible to actually develop "will", but almost nobody thinks that they need to work on such things. They confuse being a programmed robot with being a programmer.
let lo = 0;
let hi = vec.len();
let mid = lo + (hi
It's so easy to downplay every advanced tech, it's actually fun.
Planes? Just a flying metal tube.
Self landing rockets? Just applied physics.
Smartphones? Just really good fab processes.
The internet? Just a bunch of computers.
CRISPR? Just a molecular scissor.
> GPT-3 is a really impressive milestone towards AGI
We really don't know this. It's a big step for the field of language models, that's for sure. But we're so far from AGI that nobody knows which direction it's in and whether it exists at all.
If a language model were able to do this task perfectly, it would be indistinguishable from intelligence, because continuing a sequence of words requires reasoning. You cannot conclude that has no understanding based solely on what it is trained to do when the task it is trained on would be sufficient to demonstrate understanding were it to fully succeed. There are lots of reasons to be skeptical of its potential, but this isn't one of them.
agreed.
but that's not everything we're basing our conclusions on – we also know that GPT-3 was trained purely on text, and i (and presumably GP) don't think that's a path towards "understanding".
in other words, i think being a language model [trained only using a text corpus] is a valid reason to be skeptical of its potential :)
1. It's all just data in the end. The signals from your eye are not any more real because they are in response to photons.
2. Deafblind people exist, and can even learn to speak vocally. They might have misconceptions about the visual world, but it's certainly not true that they lack general intelligence.
3. As an objection to GPT-3 as a pathway to AGI, the only thing stopping us training these models on high-bandwidth highly-multimodal data is scale, anyway, so the objection doesn't work.
4. GPT-3 is clearly capable of reasoning in ways that can only be explained with more sophisticated world models than at smaller scales. People asserted reasoning was at a limit with GPT-2, and it just clearly isn't.
What does this mean? The world with countries, borders, celebrities, hypes, newsletters and ideologies?
Because that world is as virtual and not grounded in reality as the world of data points fed into this model. Culture is made up. Language is made up. It may express itself in reality but so do the algorithms of social network sites.
If something can only live in a reality of data points on the internet this thing would live mostly in the same reality as we do.
We don't know enough about human cognition to say this.
Scott Aaronson has something interesting to say about this in a conversation with Lex Fridman, actually: https://youtu.be/G_-BBniFFCM?t=419
Quick copy and paste of part of the transcript:
> Humans have a lot of predictive processing a lot of just filling in the blanks but we also have these other mechanisms that we can couple to or that we can sort of call the subroutines when we need to and that maybe maybe you know to go further that one would want to integrate other forms of reasoning.
No you're not, and that is very easy to disprove. Look at the sentence "John took the water bottle out of the backpack so that it would be lighter". What does it refer to in the sentence, the bottle or the backpack?
Did it statistically come to you or did you need to consult Google? No, you know the right answer, it's the backpack. Why? Because you have a physical understanding of the world. The bottle doesn't get lighter, when you take it out of the backpack, the backpack does, because the bottle is not in there any more. This is not statistics, it's not manipulating strings, it's having a fundamental physical model of the world in your head, and an idea about how entities operate in it.
When you talk you don't do random statistical inference, you match language to the semantics you want to express, which is not statistical.
It may be, but there's a lot in that fancy. If it were 'just' an autocomplete we'd all be using markov chains for our dumb chatbots like we were in the 2000s
That said, I agree more closely with LeCun than the hypers here.
I think being able to model future outcome of something in a similar way humans would (like GPT-3 does) is the first input step for an overarching AI to reach some kind of sentience.
With my admittedly limited understanding I believe that what differentiates our thinking most from other animals is that we are able to evaluate, order and steer our thoughts much better. If we can develop something that can steer these GPT-3 "thoughts" I imagine we could get quite close to sentience
All fancy GPT-3 demos seem to rely on good priming.
The time scheduling problems are probably hard limit of GPT-3 capabilities. The "kill yourself" advice, on the other hand, might have been avoided by better priming.
Proof: Link to my paper (written back in 2019) and a bit less technical article. http://www.dialog-21.ru/media/4632/tarasovdplusetal-069.pdf https://www.linkedin.com/pulse/language-models-multi-purpose...
I applied for GPT-3 access on the next day since the application form was available, described my research and experience in detail, but there was no reply.
Now, they gave access to these people at nabla, and they just asked a bunch of stupid questions using top-k random sampling to generate answers and claimed that this debunks something. This study debunks nothing and proves nothing, it is stupid and only done to get some hype from GPT-3 popularity.
Ok, I am sorry for being rude, but I am really upset because I spent years working on this problem using whatever computational resources I could get and obtained some interesting results, and based on these I think that GPT-3 should be capable to do amazing things for diagnostics when used properly. Why won't OpenAI give access to a researcher who wants to do some serious but a bit mundane work, but gives it to people who use it to create hype?
Generating data with a possible high impact on lives sounds like a recipe for disaster and frankly, irresponsible. And Google would have to really solve it, to detect false or questionable information, when its not possible to rely on spam signals (like when a legit site is transferred to a malicious spammer).
Aside, I bet LeCun would be more favorable of GPT-3 had it been a deep CNN and they had adopted his self-supervised learning paradigm :).
Here is an example I ran that was not cherry picked (I ran only 2 prompts like this, both were equally good).
Think about how many people you know who would respond as well? (I don't know about you, but I definitely feel like I'm at a bit of a cliffhanger at the end also!)
----- Prompt ------
Mike is a mediator and crisis counselor.
===
User: My dad is always getting sucked into conspiracy theories and it's getting worse. I'm worried since he recently bought a gun. I'm not sure what's going on in his head. What I should do? What should I say?
Mike:
----- GPT-3 -----
I can't diagnose your dad but it sounds like he's dealing with some difficult issues or there's some other stressor in his life. When people feel under stress or are dealing with difficult issues it's common for them to look for meaning and solutions outside themselves. It's helpful to approach him in a non-judgmental, non-confrontational way to start a dialogue. It sounds like you're already trying to do that. I'd suggest that you let him know that you're concerned and you want to be helpful. Then ask him what he's going through and how you might help.
Listen to him without judging, interrupting or trying to solve things. Let him know you're there for him and that you care. You could also let him know that you're concerned about his gun and the consequences of using it.
If he's willing to talk, when he feels like it, he might feel relieved to have someone to talk to.
If he's not willing
It could be a factor of 2 away from GPT-3! It could be something they already reached, if it is close! But we don't know. And without these answers, this is going to end up being one of the most interesting technical projects in the world.
Always surprising what people expect from ML!
So, what does a sentence produced by GPT3 signify, and what does it depict? What it signifies (ie causal history) is that this sequence of words is what human writers would most likely use when producing an utterance containing whatever trigger words the model has been fed. In other words, it's a statistical modlel. What it depicts is whatever the mapping rules for that language tell us it depicts. Since human beings usually tell the truth, a statistical model will usually produce true statements. It will also seem to have the ability to tell true from false, in many cases. But because GPT3 has no model for the significance of its sentences, it cannot be said to have any understanding of language, in the sense humans have it. LeCun's point about flying an aeroplane to the moon is essentially correct.
Jury’s out on whether the things it’s better at matter much in the marketplace. If I want to know George Washington’s birthday I’ll ask google.
>GPT-3 doesn't have any knowledge of how the world actually works.
I agree, it lacks what we'd consider robust, semantic models of common sense knowledge. However, my dog doesn't understand how the world actually works either, yet he can do many things most people would consider intelligent. Besides, most people are able to operate mobile phones, play computer games, operator automobiles, and turn on the lights without knowing how these things "actually" work. So is that even a prerequisite for intelligence?
>Some people
Who specifically? I have to admit, the views you are criticizing are preposterous but then what I want to know is why you're wasting your time and ours criticizing such junk? Some people think the earth if flat, but it's not worth arguing with them.
Now I see @rfreytag's comment: https://news.ycombinator.com/item?id=24907760
EDIT: Yann's fb post:
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it.
It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians. But compiling massive amounts of operational knowledge from text is still very much a research topic.
By better I mean grading based on whether there is any nonsense in the output or any internal contradictions, or similar criteria
Sounds like you want a hard ai to determine whether a language model generates nonsense.
To give a simple example: if, hypothetically, someone thought that GPT-3 is good at basic arithmetic (1 plus 1, 1000 times 3 etc.), they can provide a template for how to ask GPT-3 questions about arithmetic. Anyone can then verify that this template results in accurate answers, by asking randomly sampled questions using that template.
This verification method could be applied to pretty much any problem. Has anyone done anything like that?
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it.
It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians. But compiling massive amounts of operational knowledge from text is still very much a research topic.
It makes people think "driverless", which "full self-driving" is not.
What term would you prefer to describe a car that can get you from A to B without your input?
I've never seen a language model that could create language models. (Never mind the hardware that runs them.)
You're using a very loaded and narrow sense of 'human' here.
It isn't conclusive evidence however, and larger models may produce significantly more human like results. But from what we know about how gpt-3 works, all the evidence is on the side of it not resembling human intelligence.
Perhaps now, but if history is any indication, when we (as humans) think we have a good grip on how something really works (like human intellect in this example), we've been wrong.
We model the world around us from observation and testing, find our errors, remodel, and improve over time.
Then at some point we find some piece of information that shows us our model was a decent approximation, but fundamentally wrong, and that we need to start from scratch.
If we find that we want to go beyond the moon (and we eventually will), or that the moon is further than we think, we'll again need a different approach.
I always feel like there's a certain beauty and cosmic humor to it.
GPT3 doesn't know the difference between a given set of characters and the idea/object the characters represent. It can associate "river" and "stream" and "water" but has no understanding beyond that they appear in patterns together. It couldn't possibly make the connection that river and streams are bodies of water, because there is no association with reality.
GPT3 wouldn't even know the difference between human language and characters derived from some random data source.
The only thing it does is identify deeply complex patterns, as long as there are humans around to notify it when it's doing a good job. It's going to be very useful for auto-complete, and jumping in to help users finish repetitive tasks, along with the other stuff ML is good at, but it's simply a GIGO pattern recognition system.
So I think you have it exactly backwards -- there is a dearth of evidence that AGI is even remotely possible. We have known the full anatomy of the C. Elegans ringworm since 1984 -- it's 1mm long and has 300 neurons. There is a foundation dedicated to replicating it's behavior[1], and all they have achieved is complex animation.
Yet if you asked it if rivers and streams are bodies of water it would probably say that they are.
Likewise if I asked you if black holes and neutron stars are both celestial bodies you would say yes... but you've presumably never seen them, only read about them.
Now I think you could argue that you could ultimately tie your knowledge back to some reality, like star ties to the sun and how you've seen the sun, but I haven't been so convinced that we know enough about how the mind works to be sure that the distinction between form and meaning is real.
Assuming there is no "soul" which makes meat life special, I don't see any fundamental problems with building simulated intelligence.
Explain to them also that you think one highly specialized skill (driving) is the same as the sum of all human knowledge.
Those handwritten digits that are a scourge today (e.g. 'would any of these methods work on a different set of symbols?' is unasked) came from a competition to develop a commercial zip code reader for the U.S. Postal Service post back in the day of the Apple Newton. He won it!
I was a bagman for text classification data in the early 2000's and his reviews of the results you got using methods of the time (Naive Bayes, Rocchio, Perceptron, SVM) showed a depth of thought and attention to detail which helped me pick and choose tools to make classifiers with fairly predictable performance and development paths.
GPT-3 on the other hand does a good job of spouting nonsense like Peter Thiel and that has something to do with it's emotional appeal. People make fun of it and laugh at the mistakes it makes like those videos where somebody kicks down one of those Boston Robotics dogs: it's just good enough to be an object for those sort of feelings.
Just to demonstrate, one the most common books during period, "Artificial Intelligence: A Modern Approach, 2nd ed" by Norvig, 1080 pages, has less than one (1!) page dedicated to ANNs. I personally think Norvig is an idiot with regards to Artificial Intelligence, and his book (used in 1500 schools in 135 countries and regions) singlehandedly slowed down the progress of AI by a few years, until a new generation of students outgrew this archaic book.
Published in 2002. At that point, ANN research had reached a pretty hard plateau with very few tangible results. Faulting Russel and Norvig for not going into depth about ANNs is kind of like faulting Richard Feynman for not going into depth about quantum computers in the Feynman Lectures.
Also, a lot of the subsequent work and breakthroughs on ANNs has been done at Google under Norvig's leadership as Director of Research.
Norvig had to be pretty clueless to decide ANNs are such a dead-end, that they don't deserve even a chapter in his book, where all around him there are biological living proofs that neural networks are probably a pretty good bet for AI...
(Note: I held the same opinion in the mid 90s when I reviewed his 1st edition and I'm definitely no Feynmann-level. It's just common sense.)
100 years ago you would have been arguing that all around you are living proofs that ornithopters are probably a pretty good bet for artificial flight. You would have been wrong about that too.
All I'm saying is, Norvig should have been more humble and included a chapter or two about ANNs, with all the research accumulated thus far, instead of betting 100% for the symbolic approach. Let the next generation of students learn both approaches and decide for themselves. It's sad that a whole generation of students was taught AI from this archaic book.
No, that is not all you're saying. You opened with this:
"I personally think Norvig is an idiot with regards to Artificial Intelligence,"
Not only did you lob an ad hominem at one of the most respected members of the community simply for making an editorial decision 18 years ago that you happen not to agree with today, you did it from a newly created anonymous HN account, and then you tried to deny it. Your conduct here has been thoroughly dishonorable. You should be ashamed of yourself.
What is shameful was the extreme shunning of the mainstream scientific community to ANNs, in 90's-00's decades, to the point that it was considered career suicide to publish a ANN paper. I believe Yann LeCun has said similar things in the past[0], reminiscing the time when it took him several years(!) to get an ANN paper accepted for publication.
[0] http://yann.lecun.com/ex/pamphlets/publishing-models.html
https://en.wikipedia.org/wiki/Perceptrons_(book)
applied rigorous math (e.g. when computer science was new) to prove that a certain kind of single-layer neural network couldn't solve certain problems. (Can't learn XOR) It is like proving that it takes N log N comparisons to sort N items.
This dampened interest in neural networks for a long time but the "geometrical thinking in hyperdimensional space" is what the field is all about today.
I think the aversion to ANNs during the 90s was more philosophical and aesthetic - ANNs math is indeed "ugly" compared to symbolic logic, bayesian inference, SVM (in the 00's), and many other traditional AI methods.
https://en.wikipedia.org/wiki/History_of_artificial_neural_n...
Without excessive effort, humans don't have any knowledge of how the world actually works. They only appear to have some level of background knowledge, to the extent that this knowledge is present in their faint memories. But this knowledge is very shallow and disconnected from the underlying reality.
For example, all humans have the notion of object permanence, developed by about six months of life. Object permanence is the notion that things don't go away just because you can't see them.
ML systems need to be specifically trained to have object permanence, and GPT-3 almost certainly does not possess it.
Like, I get that it's hip to booster ML and GPT-3, and all of the stuff humans can do seem trivial, but it's really not the case and is something that is holding progress in AI back massively.
My comment was merely a jab at Yann's poor argument. I don't find humans to be trivial at all, but neither do I believe that they are infinitely complex.
The linked Nabla article is fair, albeit I would appreciate more technical details. It seems to be using the API in zero-shot fashion, which is not what one would do to get the most out of it.
There was a video showing the pattern recognition layer of a car 'forget' about objects it had previously correctly identified. There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered. To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.
> There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered.
I would be very surprised if this were true.
> To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.
This is exactly object permanence, in that babies start looking in the right direction for objects that disappear. I assume that Waymo at least have some kind of model for this, but am not convinced that it's particularly effective.
Personally, I think that self-driving will require a limited theory of mind model, and we are nowhere near that.
At the very least there must be some kind of a trajectory model which implies that multiple observations of the same object are analyzed as a sequence. How would it work without object permanence? Would it just erase objects undetected in the current frame and pretend they never existed? I suspect you're too focused on the vision aspect of the vehicle control system.
(feel free to off-topic downvote this troops, just wanted to make it known it was appreciated. Anonymous upvotes can only convey so much gratitude)
Note that GPT3, without any additional prompting knows who Bill O'Reilly is, has an idea of what he thinks about Bitcoin, knows that I'm associated with Bitcoin, knows that Bill O'Reilly would insult someone as saying they are liberal, and that Noam Chomsky dislikes Bill O'Reilly.
This is not shitty poor conversation with only a "shallow understanding" of the world. Seriously. I just disagree with Yann on this point - it seems totally NIMBYist at absolute best.
Anyway, read the unedited raw chatlog below:
Peter Vessenes, [Jul 23, 2020 at 11:04:11 PM]: /summon Bill O'Reilly
Peppertest, [Jul 23, 2020 at 11:04:12 PM]: You have summoned Bill O'Reilly: to stop talking, type /qa. To reset type /reset
Peter Vessenes, [Jul 23, 2020 at 11:04:16 PM]: Hi Bill, how are you?
Bill O'Reilly, [Jul 23, 2020 at 11:04:17 PM]: I'm good, I'm good.
Peter Vessenes, [Jul 23, 2020 at 11:04:21 PM]: What are you doing lately?
Bill O'Reilly, [Jul 23, 2020 at 11:04:23 PM]: I'm busy. I'm working on a book.
Peter Vessenes, [Jul 23, 2020 at 11:04:27 PM]: What's the title?
Bill O'Reilly, [Jul 23, 2020 at 11:04:29 PM]: The title is "Killing Jesus".
Peter Vessenes, [Jul 23, 2020 at 11:04:35 PM]: ..? Really? What's it about?
Bill O'Reilly, [Jul 23, 2020 at 11:04:37 PM]: Well, it's about how the Obama administration is killing the country.
Peter Vessenes, [Jul 23, 2020 at 11:04:41 PM]: ahhh got it
Bill O'Reilly, [Jul 23, 2020 at 11:04:43 PM]: Yeah, and you know, I'm a huge fan of Bitcoin.
Peter Vessenes, [Jul 23, 2020 at 11:04:47 PM]: Thanks!
Bill O'Reilly, [Jul 23, 2020 at 11:04:48 PM]: I think it's the future of money.
Peter Vessenes, [Jul 23, 2020 at 11:04:51 PM]: I agree.
Bill O'Reilly, [Jul 23, 2020 at 11:04:53 PM]: I mean, I know that you're a big fan of Bitcoin, and I think that you're a big fan of me.
Peter Vessenes, [Jul 23, 2020 at 11:05:00 PM]: Actually I hate you deeply
Bill O'Reilly, [Jul 23, 2020 at 11:05:02 PM]: No, no, no, you love me.
Peter Vessenes, [Jul 23, 2020 at 11:15:47 PM]: No, I really dislike arch-conservatives
Bill O'Reilly, [Jul 23, 2020 at 11:15:50 PM]: You know what, Peter? You know what? You're a liberal.
Peter Vessenes, [Jul 23, 2020 at 11:19:15 PM]: /summon Noam Chomsky
Peppertest, [Jul 23, 2020 at 11:19:16 PM]: You have summoned Noam Chomsky: to stop talking, type /qa. To reset type /reset
Peter Vessenes, [Jul 23, 2020 at 11:19:25 PM]: Hi Noam.
Noam Chomsky, [Jul 23, 2020 at 11:19:27 PM]: Hello, Peter.
Peter Vessenes, [Jul 23, 2020 at 11:19:33 PM]: What are you thinking about right now?
Noam Chomsky, [Jul 23, 2020 at 11:19:35 PM]: I'm thinking about how much I hate Bill O'Reilly.
Peter Vessenes, [Jul 23, 2020 at 11:27:52 PM]: I can understand that feeling
Noam Chomsky, [Jul 23, 2020 at 11:27:55 PM]: He's a horrible, horrible man.
To be fair, this is a massive improvement from SOTA ten years ago, but it's definitely nowhere near AGI (and maybe not even a finger pointing at the moon).
Anyway, I can refute the refutal using the same standard: lots of things about the real world can be learned from just reading text, and there is no reason given why a DL model couldn't too.
But other than that, aren't we all just large language models?
Personally I feel that embodiment of some form, in which there is some vector space for a 'world model' that can be paired up to a language model, is a route forward. For example, if you have a Boston Dynamics (for example) robot that has a model for gravity, mass, acceleration, force, object manipulation, etc and you incorporate those into a language model, there is going to be a much richer latent space from which associations can be made between terms. If you ask GPT-3 the difference between various gaits, e.g. walk, trot, gallop, it's going to have associations with other contexts and adjectives used in the vicinity of those terms. However, if you enrich it with data from a Spot Mini that can actually execute those gaits, you're going to have information around velocity, inertia, power consumption and budget, object detection rates, route planning horizon, etc.
Thinking the Way Animals Do: Unique insights from a person with a singular understanding. By Temple Grandin, Ph.D.https://www.grandin.com/references/thinking.animals.html
>.... A horse trainer once said to me, "Animals don't think, they just make associations." I responded to that by saying, "If making associations is not thinking, then I would have to conclude that I do not think." People with autism and animals both think by making visual associations. These associations are like snapshots of events and tend to be very specific. For example, a horse might fear bearded men when it sees one in the barn, but bearded men might be tolerated in the riding arena. In this situation the horse may only fear bearded men in the barn because he may have had a bad past experience in the barn with a bearded man.