Calculators are smarter then humans in calculating, what does he mean by that?
I don't think of ChatGPT as being "smart" at all, and comparing it to a human seems nonsensical to me. Yet here is a Turing award winning preeminent expert in the field telling me that AI smarter than humans is less (implied: much less) than 30 years away and quitting his job due to the ramifications.
If you're interested in exploring this further I can really recommend taking a look at some of the papers that explore GPT-4's capabilities. Most prominent among them are the "Sparks of AGI" paper from Microsoft, as well as the technical report from openai. Both of them are obviously to be taken with a grain of salt, but they serve as a pretty good jumping off point.
There are some pretty good Videos on Youtube exploring these papers if you don't want to read them yourself.
Also take a look at the stuff that Rob Miles has published over on Computerphile, as well as his own channel. He's an Alignment Researcher with a knack for explaining. He covers not just the theoretical dangers, but also real examples of misaligned ai, that alignment researchers have predicted would occur as capabilities grow.
Also I think it's important to mention that just a short while ago virtually no-one thought that shoving more layers into an llm would be enough to reach AGI. It's still unclear that it will get us all the way there, but recent developments have made a lot of ai researchers rethink that possibility, with many of them significantly shortening their own estimates as to when and how we will get there. It's very unusual that the people that are better informed and closer to the research are more worried than the rest of the world and it's worth keeping this in mind as you explore the topic.
This was basically the strategy of the OpenAI team if I understand them correctly. Most researchers in the field looked down on LLMs and it was a big surprise when they turned out to perform so well. It also seems to be the reason the big players are playing catch up right now.
I agree that I think they underestimated quite how useful a product could be built around just the language modeling objective, but it's still been critical for most NLP advances of the last ~6+ years.
https://arxiv.org/abs/2303.12712
While there is no scientific evidence that LLMs can reach AGI, they will still be practically useful for many other tasks. A human mind paired with an LLM is a powerful combination.
Here’s the thing: the authors of that paper got early access to GPT-4 and ran a bunch of tests on it. The important bit is that MSR does not see into OpenAI’s sausage making.
Now imagine if you were a peasant from 1000 AD who was given a car or TV to examine. Could you really be confident you understood how it worked by just running experiments on it as a black box? If you give a non-programmer the linux kernel, will he/she think it’s magical?
Things look like magic especially when you can’t look under the hood. The story of the Mechanical Turk is one example of that.
the human brain is a black box, we can certainly learn a lot about it by prodding and poking it.
>Things look like magic especially when you can’t look under the hood.
imagine we had a 100% complete understanding of the mechanical/chemical/electrical functioning of the human brain. Would knowing the magic make it any less magical? in some sense, yes (the mystique would be gone, bye bye dualism), but in a practical sense, not really. It's still an astonishingly useful piece of grey matter.
I'm not saying that you're wrong, but...
you'd have to provide a more rigorous rebuttal to be taken seriously.
AGI can exist without sapience and intelligence is a continuum. you can't just hand wave away GPT's capabilities which is why the sharpest minds on the planet are poking this new machine to work out wtf is going on.
human intelligence is a black box. we judge it by its outputs from given inputs. GPT is already producing human-like outputs.
a common rebuttal is: "but it doesn't *really* think/understand/feel", to which my response is: ...and? ¯\_(ツ)_/¯ what does that even mean?
What’s brilliant about this is that typically auto scaling metrics look like a stereotypical roller coaster track with the daily ups and downs!
That’s a genuinely funny, insightful, bespoke, and stylistically correct joke.
Tell me that that is not intelligence!
Well realistically it's like independently evolving the language processing part of our brain without forming the rest of the brain, there seems to be extra logic/functions that emerge within LLMs to handle these restrictions.
I think we'll see AGI when we finally try to build one up from various specialised subcomponents of a "brain". Of course GPT can't "think", it only knows how to complete a stream of text and has figured out internal hacks during training to pass the tests they set for it.
The real difference will be when we train a model to have continuous, connected abstract thoughts - an LLM can be used to communicate these thoughts or put them into words but it should not be used to generate them in the first place...
Everyone at that conference, including myself, have assumed we will eventually create smarter than human computers and beyond.
So it’s not a new position for people who have been in AI for a long time, though generally it was seen as an outsider position until recently.
There’s a ton of really great work done prior to all of this around these questions and technical approaches - I think my mentor Ben Goertzel was the pioneer here holistically, but others were doing good technical work then too.
I struggle to see how GPT-4 is not intelligent by any definition that applies to a human.
The only way anybody has ever come up with to measure it is test-taking - which machines can already do far better than we can. Real intelligence is creativity, but good luck measuring that.
Even assuming that definition, it begs the question of, "what is creativity?"
Well said.
Even Jim Keller (a key designer involved with a lot of major CPUs, in his interview with Lex Freidman) said that there might be some sort of magic, or something magical about human consciousness / the human soul. I agree with that.
That's something that a machine will never have.
hehe, this is typical goal post moving.
never is a long time.
Being handed all the correct solutions without the need to work for them in any way is a nightmare for artists and artisans, craftsmen and researchers, curious puzzle-solvers and Ayn Rand believers. It's pretty much a paradise for everyone else.
If anything, transformer models are closing the gap on that last bit, as they're built by taking the approach of "if we can't describe exactly how we rate and rank things, then let's shove so many examples at the model that it eventually gets a feel for it".
Like isn't that why humans are "more intelligent" than animals?
Plenty of animals can do things that humans can't do, but that doesn't make them necessarily "intelligent".
The fact that it seems trivially simple to fool and trick ChatGPT makes me feel like it's not very intelligent, but that's just me.
Obviously you can trick humans, but IMO it takes more effort than to trick ChatGPT. It just way too often makes such simple and stupid mistakes that it makes it hard for me to think of it as "intelligent".
I don't necessarily disagree, but I do think it is possible we will have AGI, even ASI, long before we have sentience in AI. Of course, I'm a little skeptical of measures of sentience, so even if I'm right it will certainly be debatable.
I don't know. My point is that if there is some objective indicator that's correlated with sentience, LLMs are probably already close to us on it, maybe even beating us on it. And if, at some point, a ML model reaches our levels at every objective measure we can think of, then we'll have no choice but to grant it is intelligent/sentient/sapient.
But sentience itself involves self-reflection, which there is no evidence LLMs do at all. When you submkt a prompt, a giant mathematical operation happens, and when it is complete, it stops. ChatGPT is not sitting there thinking "oh man, I should have said..."
For humans, time does not stop - we constantly process both sensory information and our own thoughts, and even if you cut out external stimuli via e.g. sensory deprivation tank, the brain will just loop on its own output instead, of which you'll suddenly become much more aware.
Sentience is that internal loop you point out. LLMs (today) don't have that. When you prompt for "write a tagline for an ice cream shop", there is no identity that remembers other prompts about ice cream, or which reflects on how taglines have changed over time, or anything else. The results can be astoundingly good, even intelligent, but there's no sentience.
If you somehow turned off a person after each sentence, upon waking up to the next prompt their first thought would be "that was weird, I must have passed out", and we could use fMRI to track brain activity indicating that thought. We are even more capable of inspecting LLMs, and there is no equivalent activity. LLMs start and end with the tokens going in and out, and a huge matrix that transforms them.
I'm generally an open minded, probabilities-rather-than-certainties person, but I'd say the odds of LLMs having sentience that we can't detect are about the same as the odds of a television having sentience that we can't detect: as close to zero as we can measure.
Yes, but that is arguably a trivial limitation. Nothing stops you from running an LLM in a loop and feed it its own output. Plenty such experiments are probably going on already - it's a trivial loop (and a trivial way to burn through your wallet). The problem is, of course, context window being rather small. So it's possible - by no means certain, but I'm no longer dismissing this idea - that the capability for sentience is already there in GPT-4 structurally, and we just lack the ability to sustain it in a loop long enough to bring it into the open.
> If you somehow turned off a person after each sentence, upon waking up to the next prompt their first thought would be "that was weird, I must have passed out", and we could use fMRI to track brain activity indicating that thought.
That's not what I meant by LLM iteration. When I said that time stops, I mean that for LLM, it literally just stops. If you were to step-execute a human like that, they would never notice; it's not like the brain has a separate RTC module constantly feeding it with sub-second resolution timestamps (and if it did, we'd turn that off too). Over a hour or more, the human may realize their inner perception of time is increasingly lagging the wall clock, but to keep it comparable to LLM, we'd be iterating sub-second process.
In this hypothetical, there would be no extra activity in brains that isn't there in LLMs. Step-executing a human mind doesn't freeze some abstract subprocess, it freezes photons and electrons and chemical gradients.
On the other hand, sentience just means having an experience of observing the world, it doesn‘t even need to include a concept of self. Presumably at least all mammals have this, for sure a dog has this. ChatGPT - probably not.
Humans became more intelligent due to developing oral and literary traditions that allowed the preservation and accumulation of knowledge. Everything that made a modern human "intelligent" is a direct result of that accumulation of knowledge, not some sort of biological miracle.
I have a feeling all of these things are limited by time/space/speed of light/heat/density limitations. Could be things can't get that much smarter than humans with in an OOM... tho they might get a lot more able to cooperate / delegate.
Often, people say yes. Those people almost universally cannot be convinced that a machine is intelligent. But, if they agree the brain is an organ, its not hard to convince them that the functions of that organ can be simulated, like any other.
edit. tried the same with GPT-4, doesn't look like it understand either, but can't ask follow up questions since I do not have access (and what really make the other answer so incredibly dumb is not so much that it gets it wrong the first time, but that it keeps not getting it despite the very not subtle hints): https://postimg.cc/ftWJXhtJ
This explains the various pejorative names given to LLMs - stochastic parrots, Chinese Rooms - etc, etc, etc.
We have LLMs that can perform "read and respond", we have systems that can interpret images and sound/speech - and we have plugins that can connect generated output to api calls - that feed back in.
Essentially this means that we could already go from "You are an automated home security system. From the front door camera you see someone trying to break in. What do you do?" - to actually building such a system.
Maybe it will just place a 911 call, maybe it will deploy a tazer. Maybe the burglar is just a kid in a Halloween costume.
The point is that just because you can chain a series of AI/autonomous systems today - with the known, gaping holes - you probably shouldn't.
Ed: Crucially the technology is here (in "Lego parts") to construct systems with (for all intents and purposes) real "agency" - that interact both with the real world, and our data (think: purchase a flight based off an email sent to your inbox).
I don't think it really matters if these simulacra embody AGI - as long as they already demonstrate agency. Ed2: Or demonstrate behavior so complex that it is indistinguishable to agency for us.
That feels fundamentally different than a calculator.
Let’s not conflate knowledge with intelligence though. GPT-4 simply isn’t intelligent.
"Here is a random string of 32 characters:
a8Jk5pYr0Dm9Nc1Vz8Qf2Bt6Hg3Lw4Uo"
I’ve tested this with a wide variety of number inputs and it’s performance is highly variable. Error also increases linearly with strong length.
does a 4 year old have intelligence?
Logic, arithmetics, algebra, precisely following steps of an algorithm - those are not skills one "kinda" just "gets" at some point, they're trained by deliberate practice, by solving lots and lots of problems specifically constructed to exercise those skills.
Point being, get GPT-4 through school, and then compare with adult performance on math-adjacent tasks. Or at least give it a chance by prompting it to solve it step-by-step as a problem, so it can search closer to the slice of latent space that encodes for relevant examples of similar problems and methods of solving them.
Anecdote: admittedly, I'm autistic as are the people I know, so maybe that's not a good sample. I struggle with a lot of basic shit even as an adult. Oh god, I empathize with the hypothetical GPT5.
GPT4 will produce stuff, but only if prodded to do so by a human.
I recently asked it to help me write some code for a Garmin smartwatch. The language used for this is MonkeyC, of which there isn't a huge amount of examples on the internet.
It confidently provided me with code, but it was terrible. There were gaps with comments suggesting what it should do, bugs, function calls that didn't exist, and many other problems.
I pointed out the issues and GPT4 kept apologising and trying new stuff, but without any improvement. There wasn't any intelligence there; the model had just intuited what a program might look like from sparse data, and then kept doing the same thing. It didn't know what it was doing; it just took directions from me. It couldn't suggest ideas when it couldn't map to a concept in memory.
A human with an IQ of 80 would know if they didn't know how to code in MonkeyC. If they thought they did, they'd soon adjust their behaviour when they realised they couldn't. They'd know where the limit of their knowledge was. They wouldn't keep trying to guess what functions were available. If they didn't have any examples in memory of what the functions might be like, they might come up with novel workarounds, or they'd appreciate what program I was trying to write and suggest a different approach.
Presumably we'll make progress on this at some point, but I think it'll take new breakthroughs, not just throwing more parameters at existing models.
What makes it even more frustrating is to iterate, you constantly have to keep it updated with any changes you made outside of chatgpt.
Don't get me wrong, it's pretty useful but it is far from a silver bullet. Getting that last 20% (or even 30%) is going to be a lot of work...
The intelligence of something is inconsequential. What truly matters is its ability to convincingly imitate intelligence.
In that sense, because we are making progress on producing an indistinguishable imitation... you might as well say we are making progress on an actual sentient intelligence.
….
in my 42 years on this planet I don't think i've made any novel discoveries.
What does a phrase like "GPT-4 scores 90th percentile on the Uniform Bar Exam" mean to you, regarding whether humans can easily surpass its knowledge and reasoning?
https://www.forbes.com/sites/johnkoetsier/2023/03/14/gpt-4-b...
Absolutely nothing, because of construct validity. Those tests measure things that have shown to correlate with abilities of concern in humans, and so are, for their purposes, valid for humans.
This hasn’t been demonstrated for LLMs, and the assumption that construct validity can be assumed without being established is begging the question: it is presuming not only that LLMs are general intelligences, but thaf they are general intelligences structurally similar to human intelligences such that the proxy measures for cognitive capacities work similarly.
I suppose, when GPT-4 writes correctly working code that does what you want on the first try, this says absolutely nothing about its cognitive capacity, because, after all, it's just a proxy measurement for the underlying generative process. (Yes, obviously the cognition is _different_ from what happens in humans. That does not mean that... it isn't intelligence?)
It says something about its ability to write code. Beyond that... its impossible to say.
We simply don’t have the information about generative AI models to be able to generalize from limited proxies about them; psychometry is not transferrable from humans to them — or at least, we have neither evidence nor a strong theoretical reason to think it should be.
When you speak to someone with an 80 IQ do they introduce themselves by saying "Hello I have an 80 IQ, nice to meet you." So that, like the person I responded to above, you can compare their conversation skills to the ChatGPT4 conversation skills?
Secondly, I'd say you're likely the one missing OPs point by trying to take a mostly colloquial statement about how ChatGPT is about as informed as the bottomish X% of the population on any given topic and trying to be pedantic about it. Furthermore the real purpose of OPs point is that the X% is now a lower bound, even if X isn't 16% but 5%, it's only going to go up from here. Yes there's evidence of diminishing returns with the current architectures but there's also a lot of room for growth with newer architectures or multimodal modals.
I think most people understand OPs point without having the need to go around asking everyone what their IQ is. There are numerous indicators, both formal and informal, that indicate that ChatGPT is as informed on most any given topic as the bottom 16% of the population. In fact, it's likely much much higher than that.
IQ of GPT is general in a sense that it can solve novel tasks that some IQ 80 individuals would not be able to as long as the tasks and responses can be encoded in plain English.
It's kind of breathtaking that we forgot about that being hard.
The goalposts are moving again.
BTW, it has passed many standardized tests under the same circumstances as a human.
No, it hasn’t, and it is physically impossible for it to. The extent to which the differences are material may be debatable, but this claim is simply false.
My understanding of what he means by that is a computer that is smarter than humans in everything, or nearly everything.
What does it even mean for the AI to be smarter than people? I certainly can't see a way for LLMs to generate "smarter" text than what's in their training data.
And even the best case interactions I've seen online still rely on human intelligence to guide the AI to good outcomes instead of bad ones.
Writing is a harder task to automate than calculation, but the calculator example seems pretty apt.
Their training data contains much more knowledge than any single human has ever had, though. If they had equivalent linguistic, understanding and reasoning abilities to a human, but with so much stored knowledge, and considering that they also win in processing speed and never get tired, that would already make them much "smarter" than humans.
Not to mention that LLMs are just the current state of the art. We don't know if there will be another breakthrough which will counter the limitation you are mentioning. We do know that AI breakthroughs are relatively common lately.
And automation is generally cheaper and faster than human labor, but that's not a very compelling definition of "smarter" either.
But, as of right now, LLMs can't generate new knowledge or validate their own outputs. We'll need a pretty significant breakthrough for that to change, and breakthroughs are pretty unpredictable.
my bar for tech singularity is an AI that can clean a toilet.
GPT's language model is already sophisticated enough to "understand" this instruction. It's missing spatial understanding and a way to interact with the real world, but I'd be honestly very surprised if there isn't a GPT or equivalent already hooked up to cameras/motors/actuators in a lab somewhere.
within our lifetimes we'll be reading papers with titles like: "does my roomba have feelings?"
By combining contexts from different fields. People are already using it with non-English languages and it responds in that language with something they couldn't previously find in that language.
But looking up information and translating it into other languages is well within the realm of human skill. And the information it's translating came from people to begin with.
However, even at human levels of competence a tool can be superior by being faster or more scalable than humans.
We know optimization engines (like social media algorithms) can cause harm by amplifying human speech. And even without algorithmic biases, moderation is expensive. We know disinformation is easy and effective online.
Add in AI tools that can be very convincing, even if they're wrong. AI tools that have been trained on human text to hide biases and build up extremely one sided narratives.
It's not like these things are particularly difficult for human beings to do. And AI might even do it unintentionally, like we've seen with biased models trained on hiring data. But the AI tools are definitely going to do it _faster_.
Edit: changed metaphor to a more commonly known one
The number of smart people I know that are struggling to see this is astonishing me each day.
1. At times and in certain instances LLMs do produce superior output to humans.
2. There is a clear trendline of improvement in AI for the past decade. From voice recognition in Alexa to Dall-E to chatGPT. The logical projection of this trendline points to an inescapble and likely possibility that if AI is not superior now it will be in the future.
I mean the GP you replied to knows what the guy is talking about, but he just doesn't want to admit it.
If you imagine the spider chart of capabilities, it’s certain that AI will be super-human on average before it is super-human on each dimension, so even when it can replace 50% of current jobs it’s likely to have its own “cognitive biases” that seem dumb to us. I think this is a cognitive bias on our part (pattern matching instead of properly probability-weighting, maybe the conjunctive fallacy).
I regret the snark in my post but I find the “pretend not to understand someone’s clear point” rhetorical device obnoxious. I am aware of a few reasonable arguments against Hinton’s position (I don’t happen to agree with them), but they require more finesse to construct.
As analogy, the more I look at it, the more it looks like an geocentric model of solar system.
He means AGI.