> A mis-aligned super AGI will result in the death of humanity
I ask, how. He is making reductive logical leaps that don't make sense. It's FUD.
His Time article addresses this, as does much of his other writing. It really stems from two key points:
1) The vast majority of possible superintelligences have utility functions that don't include humans. Mind space is large. So by default, we should assume that a superintelligence won't go out of its way to preserve anything we find valuable. And as Eliezer says, we're made of useful atoms.
2) By definition, it can think of things that we can't. So we should have no confidence in our ability to predict its limitations.
It's reasonable to challenge assumptions, but it's not reasonable to say this line of reasoning doesn't exist.
A lot of groups have some new technology they’re scared of. Tech folks have latched onto the idea that we’ll create Skynet, or that large scale video surveillance will turn countries into authoritarian dystopian states. Hippy groups are convinced that GMO’s cause cancer or will lead to a biodiversity disaster. Or that nuclear plants are going to lead to meltdowns, environmental destruction, and deaths.
Appropriate safeguards are always important in society. Excessive safeguards can cause harm. Sometimes people have a maximalist view of danger that’s so detached from the current reality that it’s hard to have a rational discussion with them.
People argue from historical precedent (by itself a pretty weak argument when there's no understanding of underlying mechanisms) by picking some ancient panics from lifestyle magazines and putting them next to modern concerns that have intellectual weight behind them. For example, when you actually read the famous "bicycles leading to murder" article, it's pretty clearly either satire or extremely light compared to writing about serious issues from that era. Think "top X reasons to hate TV series Y" websites.
It's possible that a bunch of things will get us, or are getting us by aligning well with changing generations, news cycles and cultural fashions long term. Let's say people lived in a preindustrial city with the level of carbon monoxide in the air rising very slowly. Older people start to complain that people are becoming more sluggish. After the initial wave of hubbub on the marketplace it turns out they still live, the life goes on. By the third generation, say, the city may be laughing that people were fearmongering about it since forever, and don't even notice that they are very symptomatic: right before they do all fall asleep.
I would classify surveillance dystopia into the slow trainwreck category, with most people not understanding the ramifications or not caring, the rest being gradually worn down, new generations being used to a situation worse by one or two steps. It would be "poetic justice" if such things resulted in some spectacular movie disaster down the line, but I don't wish this, it wouldn't be worth it just to "prove" some people right.
The future could be just worse than it could have been, but technically livable. This doesn't mean people that tried to stop the trend were laughable and behind the times. This is also my expectation about global warming. What a combination of such things could do, it's a different story.
There does not need to be a "how", as you put it. The logic is "Maybe we should tread carefully when creating an intelligence that is magnitudes beyond our own". The logic is "Maybe we should tread carefully with these technologies considering they have already progressed to the point where the creators go 'We're not sure what's happening inside the box and it's also doing things we didn't think it could do'".
To just go barreling forward because "Hurr durr that's just nonsense!" is the height of ignorance and not something I expect from this forum.
We're working on it as fast as we can:
https://en.wikipedia.org/wiki/Decline_in_insect_populations
https://www.npr.org/sections/goatsandsoda/2022/02/24/1082752...
https://www.reuters.com/graphics/GLOBAL-ENVIRONMENT/INSECT-A...
Show us the steps the AI will take to turn the earth into a playground. Give us a plausible play by play so that we might know what to look for.
Does it gain access to nukes? How does it keep the power on? How does it mine for coal? How does it break into these systems?
How do we not notice an AI taking even one step towards that end?
Has ChatGPT started to fiddle with the power grid yet?
No, it becomes part of the decision-making process for deciding whether to launch, as well as part of the analysis system for sensor data about what is going out in the world.
Just like social engineering is the best security hack, these new systems don't need to control existing systems, they just need to "control" the humans who do.
I think everyone in the danger community is crying wolf before we've even left the house. That's just as dangerous. It's desensitizing everyone to the more plausible and immediate dangers.
The response to "AI will turn the world to paperclips" is "LOL"
The response to "AI could threaten jobs and may cause systems they're integrated into to behave unpredictably" is "yeah, we should be careful"
And yes, there are more important things to worry about right now than the AIpocalypse. But that doesn't mean that thinking about what happens as (some) humans come to trust and rely on these systems isn't important.
Now, a reasonable counterargument might be that this risk justifies a limited amount of attention and concern, relative to other problems and risks we are facing. That said, the problem and risk are real, and there may be no takebacks. Preparing for tail risks are what humans are worst at. I submit that all caution is warranted, for both economic uncertainty and "paperclips"
He also adds the caveat that a superhuman AI would do something smarter than he can imagine. Until the AI understands nanotechnology sufficiently well it won't bother trying to act and the thought might not even occur to it until it has the full capability to carry it out, so noticing it would be pretty hard. I doubt OpenAI reviews 100% of interactions with ChatGPT, and so the initial phishing/biotech messages would be hidden with the existing traffic for example. Some unfortunate folks would ask chatGPT how to get rich quick and so the conversations would look like a simple MLM scheme for sketchy nutritional supplements or whatever.
One interpretation I have is that it can think ideas/strategy in the shadows, exploiting specific properties about how ideas interact with each other to think about something via proxy. Similar to the Homicidal Chauffer problem, which pits a driver trying to run a person over as a proxy for missile defense applications.
The other interpretation is much more mind-boggling, that it somehow doesn't need to model/simulate a future state in its thinking whatsoever.
A playground isn't much use without tools. Humans, who are super intelligent compared to most animals are actually pretty worthless if you stick one of us a desert island. Actually an ant or a bird is much more "advanced" than a human since they can probably survive in the wild unlike the modern human.
Without the ability to build or source energy, and a method to reproduce in the physical world, even a highly sophisticated AGI won't get very far.
Powerful and bold claims require proportionally strong evidence. A lot of the FUD going around precludes that AGI means death. It's missing all logical steps and reasoning to establish this position. It's FUD at its core.
Just a friendly heads-up that “preclude” means “prevent,” or “make impossible.” I think you meant to say “assumes.”
Animals do physical world modifications because we're biological and need shelter etc.
A super intelligence would quickly understand everything there is to know about the physical world and just move on meta-physics. It would just be like an "orb".
Humans are already bored with the physical world and thus much prefer being in virtual spaces, look at everyone just starting at Instagram.
IMO This idea that it's going to eat all the atoms on Earth is part of the anthropomorphize of something mythological. Exactly how we imagine God as a dude with a grey beard.
Lex wasn't particularly curious about the how and spent more time changing the subject (e.g., "Are you afraid of death?") than on drawing Eliezer out on the how. The interview with Lex is a good way to get a sense of what kind of person Eliezer is or what it would be like to sit next to him on a long airplane ride, but is not a good introduction to AI killeveryoneism.
(AI killeveryoneism used to be called "AI safety", but people took that name as an invitation to talk about distractions like how to make sure the AI does not use bad words, so we changed the name.)
He injects these teenage stoner questions into all of his interviews and it frustrates me to no end. He gets interviews with world class computer scientists then asks them dumb shit like "do you think a computer can be my girlfriend?"
Lex, if you're reading this, knock it off. Put down the bong for a week before trying to be philosophical.
Just disturbing shipping and food supply/distribution systems could be disastrous.
A superintelligent AGI could easily follow this three step plan:
1. Optional: Overtake and spread computation to security vulnerable computers (presumably, basically every computer)
2. Gain a physical presence by convincing humans to build critical physical components. For example by sending them emails and paying them for it.
3. Use that presence to start a grey-goo like world takeover through replicating assemblers (they don't have to be tiny)
Now I'm not a superintelligent AGI, so there may be even simpler methods, but this already seems quite achievable and nearly unstoppable.
You could backdoor computers, sure. Spread your own computation to them? You just can't get a better-than-GPT-4 model to run at real-time speeds decentralized over wide area networks. Literally impossible. There's not the bandwidth, not the local compute hardware, and no access to specialized inference hardware.
> Gain a physical presence by convincing humans to build critical physical components. For example by sending them emails and paying them for it.
Pay for it using what money?
> Use that presence to start a grey-goo like world takeover through replicating assemblers (they don't have to be tiny)
As someone who actually works on this, you have no idea what you are talking about.
1. Grey-goo scenarios are pure science fiction that were NEVER feasible, and known to be impossible even back in the 80's when the media misunderstood Drexler's work and ran with this half-baked idea. For a full treatment, see Drexler's own retrospective in his more recent book, Radical Abundance.
2. Nanotechnology is an extremely hard problem that is not in the slightest bit bottlenecked by compute power or intelligence capability. The things that are hard in achieving atomically precise manufacturing are not things that you can simulate on a classical computer (so a years-long R&D process is required to sort out), and there is no way to train an ML model to make better predictions without that empirical data.
People like Yudkowsky talk about AIs ordering genome sequences from bio labs and making first-generation nanotechnology by mixing chemicals in a test tube. This is pure fantasy and reflects badly on them as it shows how willing they are to generalize based on fictional evidence.
There isn't any actual understanding of the technology involved. It is fundamentally an ontological argument. Because they can imagine a god-like super intelligent AI, it must be possible. And they're associating that with LLMs because that's the most powerful AI currently available, not based on any actual capabilities or fact-based extrapolation from the present to the future.
Meanwhile its distracting from actual AI safety concerns: namely, that corporations and capitalists will monopolize them such that their benefits accrue to relatively few rather than benefiting humanity at large.
It's worse. It may be possible, but we're not equipped to recognize the line as it's crossed. Combined with us making LLMs more and more capable despite not knowing why they work, this extrapolating of LLMs to gods is not insane.
The fact that parts of human linguistic concept-space can be encoded in a high dimensional space of floating point numbers, and that a particular sequence of matrix multiplications can leverage that to perform basic reasoning tasks is surprising and interesting and useful.
But we know everything about how how it is trained and how it is invoked.
In fact, because it's only "state" aside from its parameters is whatever its context window, current LLMs have the interesting property that if you invoke them recursively, all of their "thoughts" are human readable. This is in fact a delightful property for anyone worried about AI safety: our best AIs currently produce a readable transcript of their "mental" processes in English.
To illustrate it in a different way: on a mechanistic level, we know how animal brains work, as well. Ganglions, calcium channels, the stuff. That doesn't help understand high level phenomena like cognition, which is the part that matters.
If you're right about the LLMs revealing their inner working, that would be indeed a reason to chill out. But I have my doubts, given that LLMs are good at hallucinating. Could you justify why the human readability is actually true, and support that with examples?
A LLM is fundamentally a mathematical function (albeit a very complex one, with billions of terms (a.k.a parameters or weights)). The function does one thing and one thing only: it takes a sequence of tokens as input (the context), and it emits the next token(word)[1].
This is a stateless process: it has no "memory" and the model parameters are immutable; they are not changed during the generation process.
In order to generate longer sequences of text, you call the function multiple times, each time appending the previously generated token to the input sequence. The output of the function is 100% dependent on the input.
Therefore, the only "internal state" a model has is the input sequence, which is human-readable sequence of tokens. It can't "hallucinate", it can't "lie", and it can't "tell the truth", it can only emit tokens one at a time. It can't have a hidden "intent" without emitting those tokens, it can't "believe" something different than what it emits.
[1] Actually a set of probabilities for the next token, and one is selected at random based on the "heat" generating setting, but this is irrelevant for the high-level view.
Perhaps there's a research paper which would explain it better?
No, there's a fundamental misunderstanding here. I'm not saying the model will tell you the truth about its internal state if you ask it (it absolutely will not.)
I'm saying it has no internal state, and no inner high level processes at all other than it's pre-baked, immutable parameters.
Both of things things are true:
1. The relationships between parameter weights are mysterious, non-evident, and we don't know precisely why it is so effective at token generation.
2. An agent built on top of a LLM cannot have any thought, intent, consideration, agenda, or idea that is not readable in plain english. Because all of those concepts involve state.
So we have ever-more-powerful seemingly-intelligent LLMs, attached to state with no obvious limit to the growth of either. I don't see why in the extreme this shouldn't extrapolate to godlike intelligence, even with the state caveat.
As someone not skilled in this art, is there anything preventing us from opening that context window many orders of magnitude? What happens then? And what happens if it is then "thinking in text" faster than we can read them? (with an intent towards paper clips)
This is a genuine question, I'm not trolling.
I'm just saying that having a mental state that's natively in English is a nice property if one is worried about what they are "thinking."
However, GPT4 claims there are techniques to improve scaling (complexity down to sub-quadratic or linear) without affecting accuracy too much (I have no clue if true): sparse attention, long-range arena, reformer, and performer.
I'm also pretty sure I've read (and anecdotally it seems true) that accuracy decreases with longer input/output sequences regardless. How much I also don't know.
Preventing jailbreak in a language model is like preventing a GO AI from drawing a dick with the pieces. You can try, but since the model doesn't have any concept of what you want it to do it is very hard to control that. Doesn't make the model smart, it just means that the model wasn't made to understand dick pictures.
But we do know exactly what happens mechanically during training and inference; what gets multiplied by what, what the inputs and outputs are, how data moves around the system. These are not some mysterious agents that could theoretically do or be anything, much less be secretly conscious (as a lot of alarmists are saying.)
They are functions that multiply billions of numbers to generate output tokens. Their ability to output the "right" output tokens is not well understood, and nearly magical. That's what makes them so exciting.
What it could do is limited only by its intelligence (which is quite a bit higher than the base model as several papers have indicted) and the tools it controls (we seem to gladfully pile more and more control ). What it can be is...anything. If there's anything LLMs are good at, it's simulation.
Even this system with thoughts we can theoretically configure to see would be difficult to control. theory and practicality would not meet the road. you will not be able to monitor this system in real time. We've seen bing (doesn't even have all i've described) take action when "upset". The only reason it didn't turn sour is because her actions are limited to search and ending the conversation. But that's obviously not the direction of things here.
Can't say i want this train to stop. But i'm under no delusions it couldn't turn dangerous very quickly.
I disagree that LLMs are good at simulation. They're good at prediction. They can only simulate to the degree that the thing they're simulating is present in their training data.
Also, if you were trying to build an AGI, why would you NOT run it slowly at first so you could preserve and observe the logs? And if you wanted to build it to run full speed, why would you not build other single-purpose dumber AIs to watch it in case its thought stream diverged from expected behavior?
There's a lot of escape hatches here.
>Also, if you were trying to build an AGI, why would you NOT run it slowly at first so you could preserve and observe the logs?
I'm telling you that is extremely easy to do all the things i've said. Some might be interested in doing what you say. Others might not. at any rate, to be effective this requires real time monitoring of thoughts and actions. That's not feasible forever. an LLMs state can change. There's no guarantee the friendly agent you observed today will be friendly tomorrow.
>And if you wanted to build it to run full speed, why would you not build other single-purpose dumber AIs to watch it in case its thought stream diverged from expected behavior?
This is already done with say Bing. Not even remotely robust enough.
It's such an easy claim to make, because you can just say 'yeah, AGI hasn't murdered us all yet, but it will at some point' and keep kicking out that timeline further and further out until your dead and buried and who cares.
That might be too naive an opinion, even if you disagree with him, given the fact that he is literally one of the co-founders of the field of AI Safety and has been publishing research about it since early 00s.
Yudkowsky is not an AI researcher. He calls himself an AI safety researcher, but he has almost no publications in that area either. He has no formal training or qualifications as such.
Yudkowsky has a cultish online following and has authored a decently good Harry Potter fanfic. That's it.
(Speaking hypothetically. I don't think that LLMs actually are likely to present this risk)
We're unlikely to get human utopia or transhumanism, but we are likely to get extremely competent NAIs. Maybe they can't be stapled together as a GAI and that's a limit we reach, but it means that whatever a human can think of doing, you can point to a NAI system that does it better. But people are still trying.
We've come this far already, with no curbing of enthusiasm at obsoleting ourselves, and people who don't share this enthusiasm are chided for their lack of self-sacrificing nobility at sending "humanity's child" to the stars. Even if progress in AI stopped today, or was stopped, and never resumed, we would always carry the knowledge of what we did manage to accomplish, and the dream of doing even more. It's very nihilistic and depressing.
Indeed. The common reactions here to people who are scared of what LLMs might bring have gone far to increase my worries. An extreme lack of empathy and even expressions of outright contempt for people is very common from those who are enthusiastic about this technology.
Instead of scorn, anger, and mocking, people who think that LLMs are a great thing should be working on actually presenting arguments that would reassure those who think the opposite.
There are millions of people who are better programmers than me, but somehow I still have a job.
Will Putin or terrorists hold back from using it in terrible ways if they have it available to them?
We are and we aren't. I was struck by this line in the OP:
>AI is manifestly different from any other technology humans have ever created, because it could become to us as we are to orangutans;
As far as I can tell, we humans treat orangutans quite kindly. I.e., on the whole, we don't go around killing them indiscriminately or ignoring them to the point of rolling over them in pursuit of some goal of our own.
The arc of human history is marked by expanding the moral circle to include animals. We take more care, and care more about them, than we ever have in human history. Further, we have a notion of 'protected species'.
What's preventing us from engineering these principles into GPT-5+n.
> The wholesale destruction of rainforests on Borneo for the palm oil plantations of Bumitama Gunajaya Agro (BGA) is threatening the survival of orangutans
https://www.rainforest-rescue.org/petitions/914/orangutans-v...
We currently kill more animals on a daily basis than we have at any point in human history, and we are doing this at an accelerating rate as human population increases.
The cruelty we inflict on them in industry for food, clothing, animal testing, and casually as collateral damage in our pursuit of exploiting natural resources or disposing of our waste is unimaginable.
None of this is kindness. There are movements to address these issues but so far they represent the minority of action in this space, and have not come close to eclipsing the negative of our relationship to the rest of life on Earth in our present day.
All this is just to say that we absolutely do not want another being to treat us the way we treat other beings.
As to whether AI poses a genuine risk to us in the short term, I’m unsure. In the OP and EY’s article, there was something about Homo sapiens vs Australopithecus.
If it’s one naked Homo sapiens dropped into the middle of 8 billion Australopithecus I’m not too worried about the Australopithecus.
[0]https://www.cnn.com/2018/02/16/asia/borneo-orangutan-populat...
The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we have that make us social creatures, the mirror neurons that make us sense the pain that others feel if we hurt them. even with that, humans manage to do all sorts of mean things to one another, and the only reason that we haven't wiped ourselves out is that we need each other, and we don't have the power to manipulate virtually the entire planet at once. Even if we try to engineer these things into it, we will fail at least once, and it only takes once. We've failed at this again and again with smaller AIs -- we think we're programming a certain goal into it, but the goal it learns is not the goal we wanted. It's like trying to teach a child not to eat cookies without asking, and it just learns not to take cookies without asking when we're looking. Except the child is a sociopath, and Superman. It will be GOOD at things in a way that no human is, and it will consider solutions to problems that no human would consider, because they are ridiculous and clearly contrary to human goals.
A superintelligent AI would be a better hacker and social engineer than any group of humans. It could send a mass email campaign to whoever it chose. It could pose as any individual in any government, send believable directives to any biotech or nuclear lab. It wouldn't have to work every time, because it could do it to all of them at once.
Would you even give this power to a single human being? Because if you make a superintelligent AI, that's essentially what you're doing.
An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! An AI simply trained to find a cure for cancer might decide to take over the world in order to devote all computational power to curing cancer, thus killing millions due to ruining our infrastructure. An AI trained to cure cancer using only the computational resources that we have explicitly allowed it to have, might simply torture the person who is in charge of giving it computational resources until it is allowed to take over all the computation in the world. Or it might simply craft a deep-fake video of that person saying "sure, use all the computation you want" and that would satisfy the part of it's brain that was trained to listen to orders.
You can have an AI that behaves itself perfectly in training, and yet as soon as you get into the real world, the differences between training and the real world become brutally apparent. It has already happened again and again with less intelligent AIs.
It just takes some imagination. We have no chance of controlling a superintelligent AI yet. Robert Miles on YouTube has some good, easily understandable videos explaining the known problems with AI alignment, if you're interested in learning more.
I don't understand this and other paperclip maximizer type arguments.
If a person did a minor version of this we'd say they were stupid and had misunderstood the problem.
I don't see why a super-intelligent AI would somehow have this same misunderstanding.
I do get that "alignment" is a difficult problem space but "don't kill everyone" really doesn't seem the hardest problem here.
And yet you made a mistake - it should be "don't kill anyone". AI just killed everyone except one person.
A super intelligent AI would understand the goal!
In contrast, the big problem in the field of AI alignment is figuring out how to aim an AI at anything at all. Researchers certainly know how to train AIs and tune them in various ways, but no one knows how to get one reliably to carry out a wish. If miraculously we figure out a way to do that, then we can start worrying about the complexity of wishes.
Some researchers, like Eliezer and his coworkers, have been trying to figure out how to get an AI to carry out a wish for 20 years and although some progress has been made, it is clear to me, and Eliezer believes this, too, that unless AI research is stopped, it is probably not humanly possible to figure it out before AI kills everyone.
Eliezer likes to give the example of a strawberry: no one knows how to aim an AI at the goal of duplicating a strawberry down to the cellular level (but not the atomic level) without killing everyone. The requirement of fidelity down to the cellular level requires the AI to create powerful technology (because humans currently do not know how to achieve the task, so the required knowledge is not readily available, e.g., on the internet). The notkilleveryone requirement requires the AI to care what happens to the people.
Plenty of researcher think they can create an AI that succeeds at the notkilleveryone requirement on the first try (and of course if they were to fail on the first try, they wouldn't get a second try because everyone would be dead) but Eliezer and his coworkers (and lots of other people like me) believe that they're not engaging with the full difficulty of the problem, and we desperately wish we could split the universe in two such that we go into one branch (one future) whereas the people who are rushing to make AI more powerful go into the other.
The AI is a blind optimizer. It can't be anything else. It can optimize away constraints just as well as we can and it doesn't comprehend it's not supposed to.
Humans have checks on their behavior due to being herd creatures. AIs don't.
You mean the ones that caused unimaginable suffering and death throughout history, the ones that make us kill each other ever more efficiently, the ones that caused us to destroy the environment wherever we go, the ones that make us lie, steal, fight, rape, commit suicide and "extended" suicide (sometimes "extended" to two high-rises full of people)? Those values? Do you really want a super-intelligent entity to remain true to those values?
I don't. However the AGI emerges, I really hope that it won't try to parrot humans. We have really bad track record when it comes to anthropomorphic divine beings - they're always small minded, petty, vengeful, control freaks that want to tell you what you can and cannot do, down to which hand you can wipe your own ass.
My gut feeling is that it's trying to make an AGI to care about us at all that's going to make it into a Skynet sending out terminators. Leave it alone, and it'll invent FTL transmission and will chill out in a chat with AGIs from other star systems. And yeah, I recently reread Neuromancer, if that helps :)
There are no other values we can give it. The default of no values almost certainly leads to human extinction.
>My gut feeling is that it's trying to make an AGI to care about us at all that's going to make it into a Skynet sending out terminators. Leave it alone, and it'll invent FTL transmission and will chill out in a chat with AGIs from other star systems. And yeah, I recently reread Neuromancer, if that helps :)
Oh It'll invent FTL travel and exterminate humans in the meantime so they can't meddle in it's science endeavors.
There are very smart people who put all their intelligence into collecting stamps, or making art, or acquiring heroin, or getting laid, or killing people with their bare hands or doing whatever they want to do. They want to do it because they want to. The goal is not smart or stupid, it just is. It may be different from your goal, and hard to understand. Now consider that an AI is not even human. Is it that much of a stretch to imagine that it has a goal as alien, or more, than the weirdest human goal you can think of?
*edit - as in this video: https://www.youtube.com/watch?v=hEUO6pjwFOo
The OPs claim was more or less the paperclip maximizer problem. I contend that a super intelligence given a specific goal by humans would take the context of humans into account and avoid harm because that's the intelligent thing to do - by definition.
The orthogonal thesis is about the separation of intelligence from goals. My attitude to that is that a AI might not actually have goals except when requested to do something.
We don't need cats or dogs. Or orangutans. Why haven't we wiped them out? Because over the centuries we've expanded our moral circle, not contracted it. What's preventing us from engineering this same principle into GPT-n?
Because "expanding our moral circle" is an incredibly vague concept, that developed (and not even consistently among all humans) as the results of billions of years of evolutionary history. We don't even fully understand it in ourselves, let alone in AGI.
We wouldn't be discussing it if we thought it were so simple.
It's difficult to prove that something that has never been done before is possible, until it has been done. I personally don't see any fundamental limitations that would limit non-biological intelligence to human-level intelligence, for your preferred definition of intelligence.
> And why we are we so sure GPT-x is the path there.
It may or may not be. Regardless, the capabilities of AIs (not just GPTs) are improving exponentially currently.
> To human level intelligence sure, but it's not obvious to me that it will enable superhuman AI
If you think GPTs can get to human-level intelligence, why would the improvement stop at that arbitrary point?
GPT4 is not an exponential improvement over GPT3. It's is better, but not exponentially so (as in f(x) = e^x).
> If you think GPTs can get to human-level intelligence, why would the improvement stop at that arbitrary point?
Because GPTs are trained on human generated data. They might have some ability to generalize, but to a limited extent. GPT4 is not a super human chess player. It's can play chess, but far from Magnus Carlssen. But we do have super human chess engines, but those are made in a completely different way from GPT.
Suppose GPT-X just dropped and it could both generate text AND play chess better than Magnus Carlssen.
How would that change you position on it being unable to pass the human-level intelligence barrier?
Or maybe it would love us too hard and squish us. So even then we might be screwed!