What Is AGI-Hard
lspace.swyx.io
lspace.swyx.io
Another implication: It's not really a yes/no property, whether you have AGI or not. It's a level of intelligence. And this level is continuous. And it's also not really one-dimensional.
What does human-level intelligence mean? It's still not very well-defined. You would need to define a number of tests, which measure the performance, and the model need to perform at least as good as a human on average.
So, I assume this is what the author means by "AGI-hard": A problem which requires some model to be at least as good as a human in a wide number of tests.
But I don't think this is necessary for driving. You want it actually to be much better in some tests (computer vision, reaction time, etc) and many other tests don't really matter (eg. logical reasoning, speech recognition, or so). So the autonomous driving problem is not AGI-hard.
General intelligence isn't a claim that all humans are polymaths. It's a claim that AIs are so overfitted to succeeding in one narrow human domain like rearranging ASCII strings they fail to achieve infant-level (or even worm-level) competence in virtually all others, and that it responds to new situations by assuming they're an instance of what it has been trained to do rather than considering different approaches.
That doesn't necessarily mean that we are actually good at solving those problems or that there isn't an upper level of difficulty that we will just not be able to solve practically, but we are definitely able to approach all classes of problems in general.
By contrast, a chess ai can't do anything but solve for chess states. A diffusion NN can't do anything but create images, or something that is mathematically analogous.
We can in principle do anything any conceivable specialized AI can do, even if a lot slower or error prone. Therefor, we have general intelligence.
Such things may necessarily be unknowable. If there are classes of observations that we are unable to make, or problems that we are unable to comprehend, its absolutely plausible that they would be cognitive blindspots.
In fact there's a whole theoretical strand in cognitive science / ontology that suggests this. We perceive and act only against a model of the world tuned for fitness rather than accuracy. i.e.: we are not perceiving objective reality at any scale, but rather a mental representation of it which is tailored to fitness. This is deeply unintuitive, but of course it would be since it relates to the limits on our ability to intuit.
Survival of the fittest?
and even when we have exhausted everything there is to know in the universe and solved every type of problem, we still can't be sure that there isn't a magic gap that we can't see using our intelligence.
but this is completely a god of the gaps argument, i.e. a religious argument, and thus not worth considering until we get actual evidence of such a thing.
Edit: more precise language.
At least at a super-organism level, humanity has achieved that, in my opinion. However, I'm not sure whether any individual human can be said to meet that criteria. Perhaps some exceptional examples meet that criteria, or maybe there is some vague IQ cut-off where any human could meet that criteria if given a really long time to study a problem - but I'm fairly certain that many humans don't meet that criteria individually.
Obviously this isn't a mathematical or scientifically provable statement, just a general concept that gets a bit fuzzy at the edges.
What I would add is a lot of people seem to judge AI overly harshly compared to humans, perhaps comparing them against humanity as a super-organism or at least an above average human. Sure, ChatGPT makes weird mistakes and can't write code as well as me, but compared to many people I know, it's a far better programmer and possibly less error prone in general. Too many people are like "ha it can't solve this differential equation", but my mum doesn't even know what a differential equation is, and thought that the moon was the sun at night. At least ChatGPT knows the moon is a distinct celestial body to the sun.
What about a simple test: it means being able to create better versions of ChatGPT and StableDiffusion.
Can an AI create a better version of itself, all by itself?
Because humans are going to create better versions of ChatGPT.
So no matter the exact definition of "intelligence" or "human intelligence", the goal is simple: create a better version of itself and it'll be considered intelligent.
Can't do that? It's simply not matching human-level intelligence.
So I don't see how intelligent machines wouldn't lead to "intelligence explosion" and the "technological singularity".
This doesn't follow at all. An ant will haphazardly create a better (fitter) version of an ant simply through breeding and natural selection. Meanwhile humans cannot intentionally create 'better versions' of humans (with any degree of reliability for a given selection criteria).
There's no intelligence on earth (arguably including humans) whose goal is 'to create a better version of itself'. The goal for lifeforms is reproduction, specifically at the levels of gene and individual selection. Increases in fitness are due to the process of the environment shaping the adaptation.
So it follows that while a future GPT might well be able to duplicate or improve itself, that wouldn't directly reflect its intelligence in comparison to living intelligences. Nor would it necessarily generalise beyond domain specificity.
We haven’t gone down the genetic engineering path but we could fix issues with our DNA.
I would argue that the vast majority of technology has hurt humanity and that a large proportion of technology is simply solving problems caused by other technology.
Even so, it is beside the point - humanity has demonstrably been applying its intelligence to improving technology's capabilities, which is a fact regardless of whether it has been beneficial, and that is enough for it being a candidate measure of machine intelligence.
I realize that there's an argument saying that if the machines start improving themselves of their own volition, it would be the beginning of the end for humanity, and I do not think that it can be summarily dismissed. If it turns out to be correct, you would be vindicated! - though neither of us want that.
I know it sounds extreme, but I can’t see any way around it. Stress levels, depression and anxiety, food problems, wealthy inequality and concentration, loneliness, and more are hitting all time highs by no small margin. And a lot of this has occurred over the past 100-200 years. I think that even without further investigation, one has to admit a strong correlation to industrialization. Societal changes, I would argue, are largely driven by technological developments.
And this ignores environmental and non-human animal suffering. Orcas in the Pacific Northwest are riddle with toxins and starve directly due to human technology. The waters off the cost of India and Thailand are so hypoxic that fish must swim at the top surface of the water, which has led to dramatic changes in whale feeding behavior. This is again due to technology. Again due to technology, we have decimated forests. These numbers are from memory, but I believe we have decreased worldwide forest coverage from around 75% in the early 1900s to around 25% today (these are rough numbers from memory).
The numbers paint a pretty grim picture. Yes, a lot of medical technology is wonderful, but from everything I can see, industrialization is generally not causing harmful trends to go down. The argument isn’t that all technology is bad.
The amount of technology developed by humans is highly skewed to the past couple hundred years. In that couple of hundred years, more people are starving today, right now, than there were people alive just a couple hundred years ago. One has to ask, why is that? I would argue that our technological explosion and industrialization are key components to this. For example, there's the idea that technological development feeds inequality.
This is false, more people are in extreme poverty today but that isn’t starvation as witnessed by these people going not dying off quickly. 99% of people living a couple hundred years ago where living in what we would consider extreme poverty so that’s basically complaining we have more people today.
People without access to modern medicine, safe drinking water, sanitation facilities, and education used to more or less just be people.
* 828 million people are hungry
* 345 million are experiencing acute hunger
* 924 million experience severe food insecurity
* 2.3 billion people experience moderate or severe food insecurity
* at least 9 million people die of starvation every year
https://www.fao.org/newsroom/detail/un-report-global-hunger-...
https://www.concernusa.org/story/world-hunger-facts/
https://www.who.int/news/item/06-07-2022-un-report--global-h...
https://www.un.org/en/chronicle/article/losing-25000-hunger-...
World populations:
* 1850: 1.2 billion
* 1800: 959 million
* 1750: 629 million
https://www.wolframalpha.com/input?i=world+population+1750
Wealth concentration has dramatically increased since the 1970s and 1800s.
“As many as 828 million people were affected by hunger in 2021”
This is not equivalent to saying 828 million people are experiencing hunter at the same time. Your conflating the two is exactly the way these statistics get severely distorted.
“Figures on actual starvation are difficult to come by, but according to the Food and Agriculture Organization, the less severe condition of undernourishment currently affects about 842 million people, or about one in eight (12.5%) people in the world population.[6] “ https://en.wikipedia.org/wiki/Starvation
Inequality isn’t the same as starvation, in the last 40 years inequality grew while starvation dramatically declined. “In 40 years, the proportion of malnourished people in the developing world has been more than halved. The proportion of starving people has decreased even faster.”
> Hunger: an uncomfortable or painful sensation caused by insufficient energy from diet. In this report, the term hunger is synonymous with chronic undernourishment and is measured by the prevalence of undernourishment (PoU).
> Inequality isn’t the same as starvation
Of course it isn't. They're correlated, but I only mentioned that in support of my comment a level above.
> in the last 40 years inequality grew while starvation dramatically declined. “In 40 years, the proportion of malnourished people in the developing world has been more than halved. The proportion of starving people has decreased even faster.”
Where is this coming from?
I'm not sure what your point is. The numbers are what they, and the things are defined. There's not misinterpretation. If you get a different sentiment than "industrialization has increased suffering", then that's fine, but it isn't clear where that comes from.
It’s from the same Wikipedia article, which makes it clear starvation is a much smaller number of people than existed in the word in 1750 or even 1000.
Again chronic undernourishment isn’t starvation. There a bunch of statistics saying ~850 million people regularly don’t get sufficient quantities of nutrition and calories for ideal heath is vastly different than starvation. My father was undernourished growing up in the US. He gained weight after joining the Army and was shorter than he probably should have been, but that describes the vast majority of humanity through most of human history.
Even today people in the US regularly skip meals and go hungry which is a real problem that’s being addressed by real programs that are critically important. However, it’s extremely rare for people to actually starve here and we need to remember starvation is a much worse situation.
So my point is this:
The human body can adapt to less than ideal number of calories it’s a fairly wide range. Starvation is below the threshold of survival there is no way to adapt to it and it can quickly result in death. Misrepresenting what’s going on as starvation has real world consequences because preventing starvation related deaths is far more achievable than improving the diets of 850 million people.
The actual number of starving people is relatively small which makes it an addressable problem.
But also: Once you have some model or algorithm for intelligence, where you just need to scale up the size / training data, is it really such a big intellectual accomplishment to just scale it up?
But what exactly is your definition of AGI or AGI-hard now? That something can improve itself? But what does that mean? Does a std::vector count, as it can resize itself when it needs more memory? Or wide set of intelligence tests?
So, is ChatGPT then already human-level intelligent? You can ask it how to improve itself. I'm sure the answers it will give will actually improve it.
If AI >= highest level of human intelligence, then AI > human intelligence. It is less accurate because it will miss any AI that achieves a precisely human level of intelligence, but it is probably also an easier test to define than others.
Of course simply increasing the scale of the model does not count as an improvement, there is nothing novel in that suggestion. But you are right that it opens a can of worms as to what an improvement is. Does an improvement count if it improves the model in the 80% of scenarios that only appear 20% of the time, while similarly reducing its performance in the 20% of scenarios that appear 80% of the time?
> You can ask [ChatGPT] how to improve itself. I'm sure the answers it will give will actually improve it.
Personally, I'm skeptical, but at least it is testable.
It could help the curious person learn a bit, but wouldn't add anything that wasn't already known.
Maybe it'd say: "give me more and more accurate training data, remove bad biased data". True but useless, already known. Or to a runner, how to get better: Practice more running, this and that exercise. True but nothing new.
> How can you improve GPT3 or ChatGPT?
Answer:
> There are several ways that GPT-3 (Generative Pre-trained Transformer 3) and ChatGPT could be improved:
> 1. Increase the size of the model: GPT-3 is already one of the largest language models available, but increasing the size of the model could allow it to perform even better on tasks such as language translation and summarization.
> 2. Train the model on more diverse data: GPT-3 is trained on a large dataset, but it could be improved by training it on a wider range of data sources, including more diverse languages and dialects.
> 3. Improve the model's ability to handle long-range dependencies: Language models like GPT-3 and ChatGPT are sometimes limited in their ability to understand and generate text that contains long-range dependencies, such as references to events or characters that were introduced earlier in the conversation.
> 4. Enhance the model's ability to handle out-of-vocabulary words and rare words: GPT-3 and ChatGPT can sometimes struggle to generate coherent text when they encounter words that are not in their vocabulary. Improving the model's ability to handle these cases could improve its overall performance.
> 5. Train the model on more tasks and domains: GPT-3 and ChatGPT are trained on a wide range of tasks, but there is always room for improvement. Training the model on more tasks and domains could make it more versatile and able to handle a wider range of inputs.
It is probably right with every point. So it can improve itself.
Yes, none of this is really novel. All this is basically well know. But so what? This is still a good answer to the question. TacticalCoder suggested this test. I'm not arguing that this was a good test. Actually I already implied in my earlier answer that I don't think this is really a good test for AGI or human-level intelligence.
Requiring AIs themselves to innovate in the field of AI would be a stringent test - probably too stringent as a definition of AI, though equally one that would leave little room for doubt that it had been achieved - and all this discussion shows is that TacticalCoder's outline of a test was not sufficiently guarded with caveats to avoid being open to misinterpretation.
probably not without lots of years of education, no.
Can you?
That’s kind of a silly semantic argument though. It also assumes that everyone interested in AGI is trying to make “human-level” intelligence.
So my argument would be that:
* not everyone interested in AGI is interested in replicating human intelligence,
* human-level intelligence is undefined,
* there’s an implicit assumption by your paraphrasing (quoting?) of LeCun that assumes human intelligence is the most general we know of, which is something we don’t have the ability to know.
So I think AGI is fine unless you’re explicitly trying to mimic human intelligence.
In the post, when arguing about "AGI-hard", this is very relevant. For a truly-general intelligence, it makes sense. But I think most people would not understand "AGI" this way. For some more narrow general intelligence, I don't think "AGI-hard" makes sense.
Driving needs some intelligence which is a bit generic but still different to other general intelligences, so it is not AGI-hard.
For automated driving to be acceptable within society it is going have to be at least as good as an average human driver.
In an urban street most non-speeding human drivers will see a child with a puppy off the lead and will understand that that child will be very likely to run after the puppy if it jumps into the road (And will most likely slow down in anticipation).
AGI, according to Wikipedia, is "the ability of an intelligent agent to understand or learn any intellectual task that a human being can".
How in general can an automated driving perform as well as a system with a human in the loop if it does not richly model human behaviour, i.e. achieve AGI?
Humans we will only recognize AGI when it can simulate all our flaws. We won't think the computer is able to see properly until it can observe the same optical illusions that we do (which are visual system flaws).
By general we mean being able to completely relate to *our* experience.
We are very biased even when it comes to say math, focusing on certain aspects and representations, which if are easily digestible for our brain, thus must be sign of intelligence. Huge numbers calculations - not intelligence, we cannot do that. Doing 100 problems way better than we do but doing mistake in 1 that is obvious to us - clearly not intelligence.
If it doesn't have the same framework for survival as we created with our neurochemistry - nope, it should be able to understand it. Even though we cannot wrap our heads around an octopus feelings.
Just looking at our understanding of say chimpanzees, shows how close minded we are regarding intelligence in general.
So basically, I agree with all you said, I'm just very convinced that most people will not assign AGI label to anything which doesn't mimic human perception (along with all of its flaws). I mean, maybe not necessarily mimic it, but at least kind of have a virtual machine knowing what how human would perceive it. Otherwise we'll think it doesn't understand the world.
"For the next 12 miles, please drive on the opposite lane of the motorway due to a motorcycle accident" next level prank.
Then there's a question of how supervised these algorithms are during training. Unsupervised is ideal, but some degree of supervised training isn't bad either and is more practical.
We already have human-level AI in plenty of areas, from chess to illustration to translation. A pocket calculator is human-level at arithmetic. Some flow-control monitoring software in a water treatment plant is human level at monitoring flow control. You could enumerate any number of skills in which software already meets or exceeds human levels, but even if you had a single AI that could do all of those things at a human level it would just feel like a bloated piece of software with a bunch of specialized modules that didn't add up to "real intelligence".
The only thing that I can think of which would suffice is human-type intelligence, rather than human-level intelligence. That is, a machine which is intelligent in a way that reminds us of humans. It's the Potter Stewart test for intelligence: "you know it when you see it". I don't know what else would end the debate.
To be clear, this is related to the argument about what AGI means, not whether that's a relevant topic to discuss, not whether we have systems which can solve useful problems, etc.
I was also a bit amused at the list of things which the author also claims to be AGI hard. But this is just a list of things we think are difficult. Physics? Calculus? Programming other AI agents?
Why not add chess to the list? The fact that we already have an algorithm that enables computers to teach themselves to play chess is not an a priori reason to exclude it.
From the article: "I think this is the sort of debate that makes good podcast fodder". So the article author admits that.
It's hand-wavy on vision. Parts of vision are not "AGI-hard". A key component of self-driving is something that can determine whether a volume of space is empty, occupied, or ambiguous. It's necessary to have something that does not report "empty" for occupied areas, and has a reasonably high accuracy rate on not emitting false reports of "occupied". There are sensor suites which can do that. LIDAR can do that. If you have data from a few viewpoints, you can do that. Musk has tried to turn it into an AGI problem, but that's his problem, not a property of the problem space. (If you look at California DMV autonomous vehicle crash reports, the successful companies have solved "is that area empty", and are now struggling with "what is that other car/pedestrian/bicycle going to do?")
What's inherently hard is always a good question. Aristotle thought that the human ability to do arithmetic was a key indicator of intelligence. Now we know just how few gates it takes to do arithmetic. A big surprise, and shock to many, has been the discovery that punditry, and what passes for intellectualism in some quarters, can be done by a really good autocomplete with a big training set. This says more about human discourse than it does about AI.
As I've pointed out before, we're still nowhere on "common sense", defined as not screwing up in the next 30 seconds. This is a big problem now that there are systems able to produce huge volumes of plausible blithering that's factually wrong.
That describes large parts of the internet, and we trained these models on that, so what did we really expect? ;-)
> This is a big problem now that there are systems able to produce huge volumes of plausible blithering that's factually wrong.
If because of some law of nature ChatGPT were the last iteration of automatic text generation we could ever create, it would be a big and permanent problem. Right now it’s just a big and temporary problem, with a huge opportunity for automatic fact-checking. (Or more precisely, validity and soundness checking.) Maybe the answer to the bullshit factory is a rebalancing of Brandolini’s law.
My only point is that if we’re approaching a local maxima with purely statistical approaches, it raises the stakes for symbolic approaches to come through.
Isn't the latter part of vision the part that's "AGI" hard though, and why the article talks/handwaves about "perfect visual reasoning" and not basic classification and object tracking/predicting? LIDAR can determine there's an obstruction in the road, driving AI can determine that the obstruction appears to consist of people and their intent is not possible to predict with any confidence based on observations in its training set, a human applies higher level reasoning to infer - without having seen a visually similar group of people before - that they're protestors showing demonstrable hostility to vehicles that should be avoided, a rabble of drunks pouring out of a neighbouring venue that may part in response to forward movement and judicious use of the horn, or an organized event which probably has a set route and finishing time which can be ascertained by stopping to speak to them. Plus, of course, it can handle the non-vision based bits of that scenario without having done so before.
> A big surprise, and shock to many, has been the discovery that punditry, and what passes for intellectualism in some quarters, can be done by a really good autocomplete with a big training set. This says more about human discourse than it does about AI
I strongly agree with this. It's akin to Moravec's paradox. Turns out a machine can give mediocre-undergraduate level responses to complex abstract questions about human psychology purely by pattern matching what people have written (which isn't dissimilar to how mediocre undergraduates approach the question...) whilst possessing less actionable insight into human emotion than the average dog. And we've managed that without having a clue how to make a computer happy or angry.
> Isn't the latter part of vision the part that's "AGI" hard though
Those accidents with self driving where they ram stationary objects at full speed because they cheaped out on LIDARs and only have visual feeds shows that visuo-spatial ability seems to be AGI-hard.
the claim here is that applying physics/math/etc would be AGI-hard, because it involves the AI perfectly modeling a world it doesnt live in (the simulation problem) or understanding relationships in concepts without making jumps (obeying logic).
When I say our measurement capabilities are limited, I mean that there are subsystems which are fundamentally chaotic. Small measurement inaccuracies will lead to large prediction inaccuracies, which in my view has little to do with intelligence.
I think the argument in the article is rather the other way around, ie that intelligence DEPENDS on a kind of simulation to develop proper intelligence. For instance AlphaZero was able to "solve" chess, go, etc, because it could simulate the game state perfectly.
And, the "hard" part comes from the fact that perfect simulation is impossible (with or without AI).
For an AI that will never interact with the physical world, this makes some sense.
However, as I argued in my other response, while some kind of World Model (ref LeCun) may indeed be necessary for AGI, it doesn't need to be perfect. Predictive about essential aspects (those relevant for decisions and actions) is enough.
https://syncedreview.com/2017/02/25/yann-le-cun-predicting-u...
I would argue that exactly the same goes for the human brain. Consciousness seems to play the part of the World Model for us. Between the quantum wavefunctions of the world we live in and our conscious experience (or even our sub-conscious data processing) there are layers-upon-layars-upon-layers of data loss and data compression/abstraction. Our sensory endpoints probably receive more raw data in a second than our consciousness processes in a year.
The reason it still works so well, I would argue, is a mix of darwinian learning that has been going on for hundreds of billions of years before humanity even appeared, where a data pipeline has been created in our hardware. This, combined with our ability to learn by interacting with the real world.
Creating an AI with a "perfect" world model is impossible. It will require more compute power than exists in the universe. Rather, AGI will need a simpler world model that captures the parts that are essential for its functioning, and that can be automatically updated using its input data (ideally sensors).
Just like us.
And, it seems to me, this is already happening. A self driving car will create an approximation for a 6-dimensional state for all objects (position and velocity), possibly with acceleration, speculation about intent or other "mental states. It will then extrapolate this a few seconds (or more) into the future, while keeping track of the confidence of the predictions.
This is a similar kind of world model that human brains seem to use, at least for the purpose of driving. Clearly, our brain needs a more complex model, since we make predictions much further into the future (such as the education of our children, or beyond) and we also interact with the world in more ways.
When AI systems start to make use of world model of the same complexity (type and level) as our own, and is able to "train" it both "genetically" and by standard "deep learning", I suspect it will be increasingly good at displaying what we consider "common sense".
And juding by recent developments, this could happens sooner than many think. If we're able to combine the features of ChatGPT, Tesla Self Driving, Stable Diffusion and perhaps a few other in a single model (that takes both visual data, sound and text as input, and which constructs a wold model that combines physical and abstract/textual/behavioral elements), this could happen within 10 years.
No matter how big computationally the AI is, the test was always trying to deal with something new, outside its comfort zone. Preferably as far outside it as possible. To do that successfully requires general problem solving and abstraction skills.
Of course one can do better or worse at that, including humans.
You cannot precompute most of the answer to a truly open question.
Chess AI is nowhere close to this. They easily beat humans, yes, but in the game-theoretic sense chess is not solved.
We should get used to this content creator thingy.
NP-Hardness and NP-Completeness are rigorously-defined concepts. By proving a problem NP hard you establish some extremely important properties about its nature and its relationship with other problems.
AGI-Hard sounds like the same sort of thing but really isn't. It's fundamentally just pure speculation. Of the three letters "AGI" it's arguable that we don't have a hard definition of what constitutes any of them. Then further to that we don't have any kind of idea what being in the set of AGI-Hard problems might mean other than you're in the set. Do these hypothetical AGI-Hard problems have some relationship with each other? Whereas we know P is in NP, do we have an equivalent class to P and what would that mean other than "The class of problems solvable by AGI"?
You can see this weakness when you look at the proposed examples of what is purportedly AGI-hard. What do they have in common? Mostly just that they are weakly-defined themselves. For example. How would you judge if an AGI derived from first principles something that isn't calculus but is equivalently important/innovative? Well you'd first need to define what that equivalence would mean.
But that's not what he's talking about, it's not the point. The scary thing and the important point in the article is that most of the press and non-technical people are being sold the idea that nothing is AGI-hard or requires Human Intelligence, whatever that is.
Large swaths of press & so called "tech" people in business think all these hard problems are already solved. And they use this to seek investment or pump up companies in weird ways, only to later be mystified when the AI success doesn't appear to work like they thought.
It's like arguing about whether something is NP-complete or not when a bunch of businesses are taking lots of money from investors with a pitch that makes it sound like they have figured out a way to efficiently solve all problems that were previously thought to be NP-complete. But they've shown no evidence they can do so.
wow, you phrased it better than i did. thank you!
For example, the Wikipedia article on AI-completeness mentions Bongard problems and Autonomous driving as examples of problems that might be AI-complete.
OK, so if I have an AI drives autonomously, is there some known querying strategy that I can use to make it solve Bongard problems? Can a Bongard problem-solving AI be made, by some known procedure to drive a car?
Without such reductions, at least the analogy to NP-hardness is incomplete. I believe these reductions are precisely what makes NP-hardness such a useful concept; even though we still haven't proven that any of these problems are objectively "hard," we are still able to show that if one of them is hard, then the others are as well!
One shortcoming of the analogy is that we have methods to prove when a problem is NP-hard. Are there ways to prove a problem is AGI-hard? Can it even be rigorously characterized? Relying on someone asserting it on Twitter feels unsatisfying (e.g. how accurate would experts have been at predicting the current capabilities of AI if you asked them 10 years ago? I think not very).
• "perfect visual reasoning" is not a thing, because "visual reasoning" isn't clearly defined. Nor "reasoning". Most importantly, neither is "perfect".
• It's not clear whether it's even accurate to say you need "perfect visual reasoning" for the application at hand (driving)
Determining whether something is AGI-hard is AGI-hard.
Maybe less inaccurate than you may think. Especially if you include "experts" that focus specifically at estimating future developments, and ignore "experts" that have focused all their effort into some very specific algorithmic detail.
It seems to me that a key factor in longer-term estimations is simply compute performance. If development of AI seems to be lagging somewhat compared to some predictions, it seems to me that this lag is similar to the slowdown in Moore's law. Also, for robotics, wearables and vehicles, power consumption of electronics combined with the fact that batteries as still heavy and expensive is an important limiting factor.
Now the METHODS we use to reach predictions of people like Kurzweil may be different from what we imagined. Specifically, it seems that many futurists were thinking that AGI would be reached through Turing Machines and rationalist algorithms. Instead it turns out that most progress is made through building generic learning architectures (like Transformers) and apply them at huge scale with vast amounts of data.
Similarily, a generation ago, we may have imagined we could have a single-core processor performing 1petaflop of computation in the 2020's. Instead, we have the 4090 now, that can do that many computations, but they are limited to "tensor" operations.
Now, assuming we're not making some drastic breakthrough that enables a "rationalist" type of algorithm to form AGI, we should still reach human brain-level of raw compute power sometime between 2030-2050 (which fits reasonably well with many predictions), and I would be surprised if we don't have full AGI within such a timeframe. I would give about 1:1 odds that it happens before 2040.
Still, ideas based on future computers being generic Turing Machines, just faster mean some predictions become fundamentally hard. For instance, Uploading our brains to a computer becomes a lot harder of the computer is not designed simply as a faster Turing Machine, but rather has hardware that is very specialized for some running some simple low level computation massively parallel. It may be close to impossible to run a human brain inside a GPU (at least efficiently), regardless of how many CUDA cores it has.
That means that "uploading" may require us to make a near-exact replica of an actual human brain, with it's spiking style neurons, in silicon.
Also, related predictions by people like Kurzweil may seem far fetched, such as his preditions about nanotech. On the other hand, we HAVE actually started large scale treatment of a large percentage of the world's population using nanotech medicine (mRNA vaccines). Maybe the revolution he predicted is actually about to happen. But my expectation is that it's going to be relatively slow for a few more decades.
It may become radically better if techniques like AlphaFold develop at Moore's law pace in terms of price an capability, but I suspect generic nanotech is a harder problem than AGI simply from a computational perspective.
https://arxiv.org/abs/1109.5951 gives a computable approximation (dubbed AIQ, Algorithmic Intelligence Quotient), and a reference implementation using Brainfuck programs (personally I would prefer Binary Combinatory Logic or Binary Lambda Calculus ;) )
https://github.com/google/BIG-bench - the most recent kitchen sink of a cognitive test, which includes the previous test as a subtask.
Apparently there are large language models (Chinchilla) that perform comparably to a median human level on this second test.
The graphics of the N64 where super realistic at the time and people will complain that COD games from 3 years ago looks dated.
When will graphics be done? when they are higher fidelity than what we can perceive and we can no longer distinguish under no limitations of view time and input. When will AGI be done? when a human can no longer distinguish between a computer and human without limitations on time and input.
But I guess there is no article writing left to do than is there?
An AI system is General and “adult” level if it can improve its own performance on some task without human support to be taught how to do that. In other words, can it correctly decide when it needs practice (skill acquisition) vs knowledge acquisition (filling information gaps) on its own and figure out how to acquire that (or make clear, explicit, detailed, requests for what it needs from a human and maybe negotiate alternatives if the first ask isn’t available). Right now at best we have “baby” AI where we half to spoon feed it everything and the result is coherent but nonsensical speech. Even if we made it sensical though, that would fail the generality piece unless the AI could guide the human on what could be done to make it better (and level above that would be the AI exploring doing that on its own).
Can it correctly distinguish knowledge learned vs verified and figure out when learned knowledge should be ideally be verified to double-check the quality of the knowledge. In other words, can it acquire a skill it wasn’t programmed for from scratch without losing its ability to perform similarly on other tasks?
An example of a concrete problem would be can it go and analyze a bunch of academic papers looking for obvious fraud but also find when an entire field is based on unreproduced and shaky/conflicting results?
Of course, “AGI-Hard” problems can be solved in one of two ways. Humans building more and more capable AI systems that can chip away at problems or AGI systems building more general, smarter, faster AI systems. The whole dream of the singularity is that we build the latter because that basically builds a second intelligent life form we can converse with on some level (although of course we likely won’t be able to understand anything it tries to explain to us that’s sufficiently complex for the same reason humans can’t understand the chess moves that AI engines are making anymore).
The thing is, most people can't. In fact, if you reinvented calculus yourselves, you should be very proud of your mathematical ability.
It’s not like asking if Stable Diffusion or ChatGPT can do it. Maybe there is specialized commercial AI that aren’t so widely known.
There exist right now people with low levels of physical and mental capabilities that current state of AI knowledge surpasses it by miles. We don’t go around saying “AI is better than humans”, even though - technically - it can be in some selected cases.
Another example is that the article says "It would be hard for an AI to compose this image as it has lots of different elements/emotions" - but see the Chat-GPT example below:
> Me: Can you generate a prompt for an AI to generate an interesting image?
> GPT3: Sure! Here is a prompt for an AI to generate an interesting image: "Create a surreal scene featuring a giant, humanoid robot standing on a desolate, alien planet, with a small group of humans huddled fearfully at its feet. The robot should have a sleek, futuristic design, and its glowing red eyes should convey a sense of intelligence and power. In the background, there should be a faint hint of a cityscape, suggesting that the robot has traveled far from its home world. Use bold colors and contrasting lighting to create a sense of drama and tension in the scene.
I can then pipe this into Dall-E 2.
Sure, it's not GPT3 drawing the image, and Dall-E 2 might struggle to draw that, but most humans would struggle to draw that too as drawing is hard. Hard things are sometimes hard.
LLMs solve natural langage and common sense, which are not necessary for discovering calculus.
Additionally, without a construct, like a body, in which to store these memories, AGI will continue to disappoint those who do.
As ChatGPT can contradict itself within a single reply, I doubt that this is the only issue.
Everything else like Chess, Text, Art, Games seems to be solvable by current AI systems. Robotics seems much much more difficult.
To me, as long as no AI manage to replicate those bricks, at least some of them, it won't be able to interact with us in ways that helps us efficiently with the outer, real, world.
Text and art are not solved. What would solved even mean for those? There are NNs which produce decent output given a simple prompt, but e.g. art is characterized to a large extent by development over time. Stable Diffusion isn't going to develop by itself.
Robotics OTOH is much simpler, but it might depend on your expectations. Look at Boston Dynamics.
For robotics, I just think of a "simple" tasks like picking up an object and moving it to some place. Haven't seen yet something like this in robotics, that looks human or animal like.
If you solve it, then you could maybe claim, ok the AI is more "clever" than a mouse. Currntly wouldn't agree on that. Of course in this area, it is also a hardware issue.
So far I have come up with high-dimensional spatial awareness (not sure how well we could get what happens in 1000 dimensions).
It might not be what you asked for but in practice "any sufficiently advanced technology is indistinguishable from magic" rule will come into play even with just quantitative advances. And we might get practical quantum computers at some point...
Control of (complex) nonlinear dynamical systems over long timescales will likely pose a problem and will require high-level unsavory shortcuts to work out. This may or may not include large-scale genetic engineering of mammals (including us) to express new characteristics later in life.
The moderate-scale genetic engineering, for example to uplift our cognitive capability or extend our lifespans could be easier for AIs, while for our own scientists it turned out to be barely tractable. Some will disagree here, noting the political red tape around the fields in question.
If we assume that humans are in AGI then AGI hard would be at least as intelligent as humans but possibly even more intelligent than humans.
I don't believe most people have "superior over humans" in mind when they say AGI-hard. They just mean a hard problem that is also in AGI.
Not to be contrary, but P ≠ NP is only conjectured and some computer scientists do indeed expect that a proof of P = NP will eventually be demonstrated.
Not me, actually, but for example a computer science author called Donald Knuth [1] thinks that P = NP and that a proof will eventually become known but even so it's not going to be terribly useful to anyone:
https://www.youtube.com/watch?v=XDTOs8MgQfg
______
[1] Yes I know. But just in case.
That's the thought experiment I like in this arena. It's not useful for identifying a time to AGI, but it makes me believe AGI is possible eventually.
Trillions of connections in the brain? Okay, we can make networks with trillions of parameters. Body with sensory information? We can add that. Emotions via an endocrine system? Separate interconnected subsystems? All workable in some form. A pre-encoded prior of a billion years of evolution? That one is harder, but if we're working in GHz rather than 4 Hz, can we get close enough to bootstrap?
In a fully materialist interpretation of the world, I don't see anything stopping AGI from being possible.
As someone who personally uses ChatGPT's general intelligence on a daily basis and has not encountered issues with its cognitive function that are serious enough to stop me from continuing to do so, I can confidently say that ChatGPT meets my definition of limited AGI.
Here's GPT's response:
No, ChatGPT is not an AGI (Artificial General Intelligence). It is a large language model that has been trained to predict the next word in a sequence of text given the previous words. While ChatGPT has been trained on a diverse range of text data and is able to generate human-like text, it is not capable of understanding or learning about the world in the way that a human does. It does not have the ability to reason, plan, or solve problems in a general way. Instead, it is specifically designed to generate text based on patterns it has learned from the data it was trained on.
These types of responses are, I believe, part of a separately trained safety system, that should kick in when people are asking things that openAI thinks chatGPT cannot/should not do. It has a lot of false positives as well as false negatives.
Long story short: I don't trust what chatGPT has to say on its own abilities.
Self driving cars: barely any fatal accidents but some erratic behavior. People go "oh we can't have that because something extremely unlikely might kill somebody. Never mind it hasn't happened yet and might never happen. But it just might. Therefore I will never trust AIs!!!!"
Self driving humans: lots of fatal accidents and most of them because of human error. People go: "that's just the way things are; price of being on the road is playing Russian roulette"
It's irrational. Trick question: Would you rather drive on a road where every other driver is an AI or on a road where the average driver is a fair selection of the current drivers? I'd feel safer with AIs probably. I risk my life regularly taking part in the traffic. It's not great. So, yes less of that would be great. IMHO that will become the norm not so long from now because of insurance fees.
So, yes, chat gpt makes mistakes and sometimes talks out of its ass. Read the average HN comment section for some human equivalent of that. Lots of supposedly intelligent people asserting things that are patently false, people applying broken logic, speaking with confidence about a brain fart they had, etc.
We're applying double standards here. The key thing for AGI is the ability to learn and improve. Chat gpt3 is not there yet. You train it once and then you deploy it with static capabilities. It never gets better. It doesn't experiment and learn from the mistakes it makes. It lacks the capability to learn. You can't even educate it about the mistakes it makes. Using chat gpt-3 does not evolve the AI model. Using your brain does evolve it. Out of the box, it's actually quite useless. Forget self driving. Just learning to walk takes years. It takes many years more to get some sensible thought out of a human being. And frankly, only very few people get very good at that.
An AGI would need the ability to absorb new information and adapt how it behaves based on that. It doesn't have to get it perfectly in one go. It just needs to be able to adapt at a similar or better pace than we do to rapidly get to the point where it is going to be better than us at just about anything it bothers to learn.
Thing is, if the roads are made so that only "AI" cars can drive on them, you don't need AGI, or an advanced AI. You basically need the cars at the level they are now (or even from five years ago). Because the number of weird shit you need to deal with is reduced significantly.
If you are also able to prevent people and animals from accessing the roads and fix the weather...
(At the other end of the scale, if you put the cars on rails in tunnels, "AI" from the 1960s will likely suffice for safe driving)
If all cars are "AI", and move in uniform fashion, and know each others' location, then this might be an easier issue to solve than it is today.
"May" is dangerously close to "just", I know :)
Fatal accidents have been rare because of human intervention. The cars aren’t self driving and no one in their right mind will stop paying attention when in one of these cars.
So the “extremely unlikely” argument is very, very weak.
The US has about 1.3 deaths per 100 million miles driven. None of the self-driving car models have driven even close to 100 million miles. Waymo is supposedly the closest with about 20 million. If they were as safe as humans, we should have seen 0-1 deaths, but there have been multiple.
Same goes for Spelunky, or Caves of Qud, or nearly any modern roguelike/rogue-lite game.
(AI hard redirected to this)
Going to feed it into ChatGPT and have it explain to me...
Hello ! I'm going to paste below a copy of a blog post on "AGI", which I don't get at all. Could you help me understand it, and/or possibly rewrite it in a way that would make it clearer or easier to understand.
Here the copy-pasted article:
<article pasted here>
The answer:
The author of this blog post is discussing the concept of artificial general intelligence (AGI), which is often described as the hypothetical future ability of artificial intelligence (AI) to understand or learn any intellectual task that a human being can. The author notes that it is difficult to have a meaningful conversation about the potential for AI to perform various tasks because the definition of AI is so broad and vague, and because it is difficult to distinguish between realistic and unrealistic predictions about what AI might be able to achieve. The author suggests that the concept of "NP-hard" problems, which refers to a class of problems in computer science that are difficult to solve and may require a very long time to complete, could be a useful way to frame discussions about the limits of AI and what it might be able to achieve. The author also discusses the importance of considering the social and ethical implications of AI, and the need to be cautious about hype and over-exaggeration in discussions about the potential of AI.
(I guess we should say "have her generate AI-tically an abstract)