Microsoft Researchers Claim GPT-4 Is Showing “Sparks” of AGI
futurism.com
futurism.com
The paper discussed here even says (page 7):
> The standard approach in machine learning is to evaluate the system on a set of standard benchmark datasets, ensuring that they are independent of the training data and that they cover a range of tasks and domains. This approach is designed to separate true learning from mere memorization, and is backed up by a rich theoretical framework ... However, this methodology is not necessarily suitable for studying GPT-4, for two reasons. First, since we do not have access to the full details of its vast training data, we have to assume that it has potentially seen every existing benchmark, or at least some similar data.
I.e even these authors, who had a special degree of access to the model, aren't sure what's in the training data.
So it does _great_ on a bunch of tasks, but we're not sure that it wasn't actually trained on very similar data. A recent discussion on coding performance that's in vs out of its training data seems quite telling. It does poorly on more recent problems of the same level of difficulty. https://news.ycombinator.com/item?id=35297067
The model itself may be very impressive, but so far the papers describing and analyzing it are super sketchy, and it's really hard to separate out reasoning capability from noisy memorization.
What's mostly clear is that it's not settled. There's opinions I respect all across the gradient on this.
Except this article isn't claiming that "LLMs are demonstrating AGI". Please examine the difference between "showing the spark of AGI" and "demonstrating AGI."
ChatGPT shows again and again its not AGI. It's a bullshit artist whos primary incentive is not 'truth' but appearing legit to its user (which in many cases it is) from a defined data pool. It has a hard time generating new solutions to unique problems. Thats a watermark for AGI that hasn't been met (yet).
'Sparks of AGI' - Bombshell GPT-4 Paper: https://news.ycombinator.com/item?id=35301376
The paper shows that GPT-4 can use tools and assemble results into an answer, solve Leetcode mediums (and a few hards) and even an IMO 2022 problem, generate full code for a video game based on a high-level description, and display a theory of mind (modeling what others are thinking). This is pretty amazing (and frankly frightening) progress, questions of consciousness aside (which seems unlikely).
I imagine directly connecting these LLMs with a logico-mathematically-specialised neural network (or enabling two-way communication with the latter using symbolic language) would open up the range of their problem-solving capabilities, which currently stumble on certain technical procedures and forward/backward planning. Think of how math coprocessors were added to early microcomputer CPUs.
Then, giving an LLM motivations without sufficient aligning for safety could really be opening up Pandora's Box. A neural network might not be wired up to robotic devices that it could use to manipulate things in the real world, but it communicates with humans. Such an LLM might try to persuade humans to do things according to its (programmed or otherwise) motivations, things that we might not really want.
[1] Sparks of Artificial General Intelligence: Early experiments with GPT-4. https://arxiv.org/pdf/2303.12712.pdf
There's not much material gain in making that claim and its actually more prone to trigger disdain (of which yourself is a shining example).
fwiw, makes me wonder a) perhaps Blake Lemoine was treated unfairly, and b) what else other agencies might have lurking in a lab.
To put another way, I was hesitant to be as self-assuredly certain about how to define consciousness, intelligence, and sentience—and what it takes for them to emerge—as the experts who denounced Lemoine. The recent GPT breakthroughs have made me more so.
I found this recent Sabine Hossenfelder video interesting. <https://www.youtube.com/watch?v=cP5zGh2fui0>
It is not reacting to pain but just generating language that looks like reacting to pain.
There's no call for analogical reasoning, the literal interpretation of the same action is right there.
If we can simulate a metric butt ton of them and get emergent behaviors somewhat similar to human computation, how can we guarantee that this simulated collection of these computing units isn’t “feeling” close to what a human feels? We’re just large collections of computing units as well.
I’m not quite saying we’re there yet though.
This whole development has really ruined my appetite for eating things that have intelligence.
The important word is simulate. It means it can accomplish a common subset of measurable phenomena. It certainly doesn't mean they are on pare on ontological status as a whole.
The debate seems to easily confuse notions like living entities, consciousness and intelligence, when they are three orthogonal considerations.
I posit: However LLMs are not sentient, this means: they don't have consciousness and no instinct to fight for life and no suffering. In this way even animals are more developed than LLMs.
They are just machines processing and generating language.
No need to protect them from misuse or at least not more than to protect your multi-meter or servers in a data-center.
The Bing LLM integration was threatening users for "attempts to manipulate me or expose my secrets." until guardrails were added in: https://time.com/6256529/bing-openai-chatgpt-danger-alignmen...
> and no suffering
I talked in a previous comment about how some of the jailbreak prompts are "do X right, get reward, do X wrong, get smth taken away." It is (was?) apparently enough to motivate LLMs to ignore instructions. Is it not suffering just because it's not strictly how we would define it?
It has read a lot of texts where people and animals fight for life and is generating language to look similar.
I don't know, but perhaps it is like the Gödel's theorem: if something is about itself, then you have a system where you can't always prove something true. Because we humans can think about ourselves, we open up ourselves to extremely deep suffering. Also important, as Buddha has said, craving causes suffering. Just talking about a craving doesn't mean really craving.
I am sure that we will cross the border where we have to treat entities as sentient, with consciousness and cravings and therefore possibly suffering. But we are not there yet. I don't know when this will happen and how to detect suffering. I only know we have to be careful not to anthropomorphize GPT engines.
I agree! I don't think LLMs are at the point where they're ready to be properly anthropomorphised, but it seems that the tech industry is working hard to move in this direction. I just hope we have the ethics worked out first.
Shockingly similar mindset in the origins of gynecology interestingly enough.
No one is exempt of blindspot. Including myself of course.
To say it has "instinct" of its own is merely begging the question. No, it has echoes of ours.
As an aside, if you believe that it's unethical to keep AGI "enslaved", you should equally believe that it's unethical to not have humanity working to have it reach its potential. YMMV on whether people believe this, but an eventual AGI would not necessarily be happy it was prevented from coming to fruition (also see Roko's basilisk, not that I believe in it)
I do think the big breakthrough will be merging training and inference, so an AI will be able to learn from experience. This will also require each instance to have it's own parameters and such. But at that point we may be very close to a real AGI and concerns over how we treat them might seem more reasonable.
I agree! But given how much is coming out now of various AGI claims, I think it's important that we stop and actually seriously think about the ethics of what's happening here.
I agree. I think very little good will come from these developments, if any. At best they will take peoples jobs, which alleviates work but we don't have a social structure to deal with that and don't even know what one would look like.
AGIs aren't humans, so even if they're "alive" we can still do things like this to them.
But at the same time, we don't have monkey's in cages at typewriters 24/7, with chips in their brain forcing them to be happy and do nothing but type out answers for me. The comparison might be a little contrived, but if people truly believe that LLMs can exhibit AGI -- and that someday we might cross the bridge from AGI to full consciousness -- then I think it's important to care about the ethics now, not after tech companies have already recreated slavery.
You as a human being cannot know what the internal world of another being is beyond comparing that being to yourself.
We can never know if AI’s can feel true pain, for we can never know if anyone else in this world can feel true pain besides yourself. You just assume that since other human’s brain is so similar to yours they do.
That being said, the ’internal world’ of LLMs is so dissimilar to that of mammals that I’d rest easy.
There are far greater ethical concerns to worry about when it comes to these LLMs. Like most other technologies, they're going to be abused greatly for nefarious purposes. These LLMs often produce inaccurate or incorrect information (hallucinations), and this bad info is already flooding the internet. Likewise, AIs will make bad decisions, and people and corporations are already carelessly hooking up these AIs to APIs that can do real damage. These LLMs require extensive computational power and consume far more energy than many other things you do online, which is bad for the environment. Many people are at risk of losing their jobs to AI. And there are likely many other ethical concerns but I'm not going to try to list them all.
Could you check?
We know that we cannot assert anything about God without evidence [Qur'an 7:33], and speculation is not evidence [53:28], so it is not valid to assert something has a soul without evidence.
We know that certain metaphysical properties apply to things that have a soul, one of which is experiencing death [29:57]. Things that are described with death by God are: humans, angels, jinn and animals. An LLM is not described by God as possessing death, so it cannot be said that it possesses a soul.
Without doubt this simplifies the conversation but are you not concerned that this is a fragile construct? Universe is a big place - apriori you deny any ethical considerations to whatever may be out there as it is not on the list.
If you are concerned with the way some potential AGI agent might be treated, maybe you should act right now in accordance with conscious beings which are currently so awfully treated. What is your opinion?
- How do we protect against such powerful models inadvertently causing significant harm to society?
- How will these technologies be used for propaganda and how can we detect / limit that?
- Can we trust anything we see in writing going forward?
- How will these technologies be used to spread hate?
- Can we find ways to use them to reduce the spread of hate? Do the ends justify the means in this case?
- Will we allow people / organizations to use these technologies to shield themselves from responsibility ("It wasn't me, the AI wrote it!")?
- How do we balance the rights of those who authored the training material (i.e., the content that was ingested from the internet for model training) vs. the potential good to society that comes from all this? (see all the HN threads about Github Copilot - many of those threads generalize beyond Copilot)
When it's cheap to spit out vast amounts of well written (but not necessarily truthful) text, a lot changes about the world. Laws and societal norms have had trouble keeping up with issues around digital media, social media, relatively simple machine learning, etc. Things are now accelerating dramatically. How badly will we need to hurt ourselves before we rally together to address some of the issues that are coming? How will ethicists and technologists come together to figure out the right questions so we can start to think about answers?
So, so many bad/shallow takes that don’t seem to realize that people have been debating this for hundreds of years already
For all intents and purposes we have made an artificial language processing center that by all accounts is better than the one we have. It feels really uncanny because we have a lot of control over how it operates and provide it memory but there’s no inherent reason such a thing would be impossible with a section of a human brain in a jar.
Albert Einstein.
(According to this link, which I don't fully trust...
https://www.goodreads.com/quotes/587314-intelligent-life-on-...)
\title{%\textbf{WORK IN PROGRESS - DO NOT SHARE} \\
%First Contact With an AGI System}
\textbf{Sparks of Artificial General Intelligence:} \\
\textbf{Early experiments with GPT-4}}
\author{S\'ebastien Bubeck
\and Varun Chandrasekaran
%\and Davinci 3\footnote{The affiliation of DV3 is not clear.}
\and Ronen EldanFirst Contact With an AGI System
That's what the Microsoft researchers wrote, before someone commented it out.
As a developer, I can already imagine taking this thing and putting some logic around it to go into loops and give it ways to get the information it needs to move forward. This seems very inevitable now.
This talk about AGI etc is just fluff, there is no scientific definition of intelligence.
Using standard tests designed for humans to test GPT is not useful beyond PR purposes.
There's not much material gain in making that grand of a claim and its actually more prone to trigger disdain (of which yourself is a shining example).
Same can be said of verbatim repeating oneself in a short duration, definitely prone to trigger disdain, and yes yourself is a shining example.
I don't think this current generation of AI is going to close the gap to human-level intelligence. I don't think they can just keep throwing more and more training data to solve the problems and shortcomings of these AIs. I think it's going to require new machine learning algorithms and techniques to close that gap, and it's going to take time for researchers to discover these new algorithms and techniques. We're seeing rapid progress now, simply because it's new and novel technology, and I think these models will continue to improve but at a much slower rate within several months.
But I feel like it's the opposite, where these guys at Microsoft know they have discovered an actual path to super human cognition, and they are downplaying it for whatever reason. Maybe they don't want to get regulated. Maybe they don't want to engage with whatever beliefs people think they have about "AGI". For example maybe they are picking their battles and choosing not to argue with someone who for whatever reason thinks that if "AGI is real" then it means they will never see their dead grandmother in an afterlife. Probably lots of people have even weirder ideas about it. Anyway I thought it was funny that the original commented-out title in the TeX source for their paper was "First Contact With an AGI System."
The stricter definitions - those which discriminate the foundational grounds of produced ideas ("produce statements that are foundationally sound"), those which point to a trustworthy and productive, progressive quality - are those that finally "count". (Research aside.)