From the notes:
"A reality check for people that think full embodied AGI is right around the corner is to ask your dancing humanoid robot to pick up a joystick and learn how to play an obscure video game."
"A reality check for people that think full embodied AGI is right around the corner is to ask your dancing humanoid robot to pick up a joystick and learn how to play an obscure video game."
From what I can tell most in AI are currently hoping LLMs reach that point quick just because the hype is not helping AI at all.
I hate Adobe, I don’t like to give them credit for anything. But their audio enhance tool is actual sorcery. Every competitor isn’t even close. You can take garbage zoom audio and make it sound like it was borderline recorded in a treated room/studio. I’ve been in production for almost 15 years and it would take me half a day or more of tweaking a voice track with multiple tools that cost me hundreds of dollars to get it 50% as good as what they accomplish in a minute with the click of a button.
It's not about specialized vs generalized models - it's about how models are trained. The chess engine that beat Kasparov is a specialized model (it only plays chess), yet it's the bitter lesson's example for the smarter way to do AI.
Chess engines are better at chess than LLMs. It's not close. Perhaps eventually a superintelligence will surpass the engines, but that's far from assured.
Specialized AI are hardly obsolete and may never be. This hypothetical superintelligence may even decide not to waste resources trying to surpass the chess AI and instead use it as a tool.
From Wikipedia, Garry Kasparov said it was a pleasure to watch AlphaZero play, especially since "its style was open and dynamic like his own".
People can't define AI because they don't want to consider AI as a subset of exponentially difficult algorithms, but they do want to consider AI as a generator of stylistic responses.
You can call this hype, maybe it is all hype until LLMs can work on 10M LOC codebases, but recognize that LLMs are a shift that is totally incomparable to any previous AI advancement.
It will help a single human do more in a white collar world.
Sounds like AI robbed you of an opportunity to spend some time with your Dad, to me
It takes five minutes to program the thermostat, then you can have a beer on the patio if that's your speed and catch up for a bit
Life is little moments, not always the big commitments like taking a day to go fishing
That's the point of automating all of ourselves out of work, right? So we have more time to enjoy spending time with the people we love?
So isn't it kind of sad if we wind up automating those moments out of our lives instead?
I'd definitely like to improve my skills, but to be realistic, most of the programmers are not top-notch.
Because we evolved to get where we are, humans have all sorts of messy behaviours that aren't really compatible with a utopian society. Theft, violence, crime, greed - it's all completely unnecessary and yet most of us can't bring ourselves to solve these problems. And plenty are happy to live apathetically while billionaires become trillionaires...for what exactly? There's a whole industry of hyper-luxury goods now, because they make so much money even regular luxury is too cheap.
If we can produce AGI that exceeds the capabilities of our species, then my hope is that rather than the typical outcome of "they kill us all", that they will simply keep us in line. They will babysit us. They will force us all to get along, to ensure that we treat each other fairly.
As a parent teaches children to share by forcing them to break the cookie in half, perhaps AI will do the same for us.
Funnily enough, I still think some of the most interesting semi-recent writing on utopia was done ~15 years ago by... Eliezer Yudkowsky. You might be interested in the article on "Amputation of Destiny."
Link: https://www.lesswrong.com/posts/K4aGvLnHvYgX9pZHS/the-fun-th...
If AGI is created it is most likely to be guided by someone like Altman or Musk, people whose interests couldn't be farther from what you describe. They want to make themselves gods and couldn't care less about random plebs.
If AGI is setting its own principles then I fail to see why it would care about us at all. Maybe we'll be amusing as pets but I expect a superhuman intelligence will treat us like we treat ants.
But it's not: many of those billionaires are using their wealth to subvert democracy. They are quite literally enemies of the state.
The only lasting defense is to prevent this level of individual wealth accumulation.
Basically, if we are going to allow single humans to have so much buying power, that needs to come with some clear expectations.
Risks must be taken. Pick something and advance society. Or, empower others in a reasonable, non discriminatory way, with no hooks.
There are a ton of things need doing.
Why not require a real effort?
We do not need more dynasties, and other enduring signs of wealth.
We do need those people to have earned it. (That is the no dynasty part again)
I know sounds harsh, but the fact is anyone can live very well for the rest of their lives on just a fraction of that money.
May well put it to work.
And if it were me personally? I got a list. Ready?
Why would you think or claim those behaviors are unnecessary?
> They will force us all to get along, to ensure that we treat each other fairly.
Well, I guess we shall see what the future brings, won't we.
AGI is really something else entirely. If this was clarified, especially for Wall Street, we would see some changes in valuations.
At a certain point, it depends on how you define human reasoning capabilities. Maybe a collection of refined and specialized AI can get close. One counter argument could be that we are not far off from highly advanced robotics that have increasingly powerful reasoning abilities even without AGI.
After the automobile was created, global horse population fell dramatically. Who knows if the wealthy classes will need all these people to do the work for them in the near future.
It would seem our own generalized intelligence is an emergent property of many, _many_ specialized processes
I wonder if AI is the same
You can say that about other animals, but about humans it is not so sure. No animal can be taught as general set of skills as a human can, they might have some better specialized skills but clearly there is something special that makes humans so much more versatile.
So it seems there was this simple little thing humans got that makes them general, while for example our very close relatives the monkeys are not.
Science is full of theories that are correct per our current knowledge and then subsequently disproven when research/methods/etc improves.
Humans aren't special, we are made from blood & bone, not magic. We will eventually build AGI if we keep at it. However unlike VCs with no real skills except having a lot of money™, I couldn't say whether this is gonna happen in 2 years or 2000.
Each of those 100 can hire teams or colleagues to make their domain better, so there’s always human expertise keeping the model updated.
200 million to have dedicated top experts on hand is reasonable.
I made a google form question for collecting AGI definitions cause I don't see anyone else doing it and I find it infinitely frustrating the range of definitions for this concept:
https://docs.google.com/forms/d/e/1FAIpQLScDF5_CMSjHZDDexHkc...
My concern is that people never get focused enough to care to define it - seems like the most likely case.
It's an ideal that some people believe in, and we're perpetually marching towards it
Can we just use Morris et al and move on with our lives?
Position: Levels of AGI for Operationalizing Progress on the Path to AGI: https://arxiv.org/html/2311.02462v4
There are generational policy and societal shifts that need to be addressed somewhere around true Competent AGI (50% of knowledge work tasks automatable). Just like climate change, we need a shared lexicon to refer to this continuum. You can argue for different values of X but the crucial point is if X% of knowledge work is automated within a decade, then there are obvious risks we need to think about.
So much of the discourse is stuck at “we will never get to X=99” when we could agree to disagree on that and move on to considering the x=25 case. Or predict our timelines for X and then actually be held accountable for our falsifiable predictions, instead of the current vide based discussions.
For me, I just zoom out a little further and say: at the rate AGI is approaching, what is the utility in trying to regulate it ahead of time?
Seems like advancement is slow enough that society can/will naturally regulate it based on what feels comfortable.
And it's a global phenomenon that can't have rules applied at the protocol level like the internet, because it's so culturally subjective.
Precedents need to be set first, and I think we'll only be able to call them when we see them.
Personally I don’t think politicians are capable of adapting fast enough to this extreme scenario. So they need to start thinking about it (and building and debating legislation) long before it’s truly needed.
Of course if it turns out that we are living in one of the possible worlds where true economically meaningful capabilities are growing more slowly, or bottlenecks just happen to appear at this critical phase in the growth curve, then this line of preparation isn’t needed, but I’m more concerned about downside tail risk than the real but bounded costs of delaying progress by a couple years. (Though of course, we must ensure we don’t do to AI what we did to nuclear).
Finally I’ll note in agreement with your point, that there are a whole class of solutions that are mostly incomprehensible or inconceivable to most people at this time (ie currently fully outside the Overton Window). Eg radical abundance -> UBI might just solve the potential inequities of the tech, and therefore make premature job protection legislation vastly harmful on net. I mostly say “just full send it” when it comes to these mundane harms, it’s the existential ones (including non-death “loss of control” scenarios) that I feel warrant some careful thought. For that reason while I see where you are coming from, I somewhat disagree on your conclusion; I think we can meaningfully start acting on this as a society now.
I like your idea of developing a new economic model as a proxy for possible futures; that at least can serve as a thinking platform.
Your comment inspired me to look at historical examples of this happening. Two trends emerged:
1. Rapid change always precedes policy. Couldn't find any examples of the reverse. That doesn't discount what you're saying at all, it reiterates that we probably need to be as vigilant and proactive as possible.
and related:
2. Things that seem impossible become normative. Electricity. The Industrial Revolution. Massive change turns into furniture. We adapt quickly as individuals even if societies collectively struggle to keep up. There will be many people that get caught in the margins, though.
Consider me fully convinced!
There is no point collecting definitions for AGI, it was not conceived as a description for something novel or provably existent. It is "Happy Meal marketing" but aimed for adults.
My masters thesis advisor Ben Goertzel popularized the term and has been hosting the AGI conference since 2008:
https://goertzel.org/agiri06/%5B1%5D%20Introduction_Nov15_PW...
I had lunch with Yoshua Bengio at AGI 2014 and it was most of the conversation that day
The term AGI is obviously used very loosely with little agreement to it's precise definition, but I think a lot of people take it to mean not only generality, but specifically human-level generality, and human-level ability to learn from experience and solve problems.
A large part of the problem with AGI being poorly defined is that intelligence itself is poorly defined. Even if we choose to define AGI as meaning human-level intelligence, what does THAT mean? I think there is a simple reductionist definition of intelligence (as the word is used to refer to human/animal intelligence), but ultimately the meaning of words are derived from their usage, and the word "intelligence" is used in 100 different ways ...
I've thought for a while that the middle letter in AGI ('General' vs 'Specific') would be more useful and helpful if it were changed to Wide vs Narrow. All AIs can be evaluated on a scale of narrow to wide in terms of their abilities and I don't think that will change anytime soon.
Everyone understands that something is only wide or narrow in comparison to something else. While that's also true of the terms "general' and 'specific', those are less used that way in daily conversation these days. In science and tech we make distinctions about generalized vs specific but 'general' isn't a conversational term like 50 or 100 years ago. When I was a kid my grandparents would call the local supermarket, the 'general store' which I thought was an unusual usage even then.
Researchers at Google have proposed a classification scheme with multiple levels of AGI. There are different opinions in the research community.
Now that the term is in the general lexicon (which is crazy to me as an old guy doing this a long time) it’s morphing into something new
Like any good scientist i want to sample the population
There comes a theoretical point at which a definition is no longer relevant because it's obvious to everyone on an intuitive level. An easy lower bound for where this threshold might sit would be "when it can start and win wars unassisted under its own volition". Since, at that point no one on earth would have a need to debate it. It would simply be respected and understood for what it is without needing to define it.
Until such an obvious threshold is crossed, it will be whatever executives, product managers, and marketing teams say it is.
Carmack believes AGI systems should be able to learn new tasks in realtime alongside humans in the real world.
Games generally are solvable for AI because they have feedback loops and a clear success or failure criteria. If the "picking up a Joystick" part is the limiting factor, sure. But why would we want robots to use an interface (especially a modern controller) heavily optimized for human hands; that seems like the definition of a horseless carriage.
I'm sure if you compared a monkey and a dolphins performance using a joystick you'd get results that aren't really correlated with their intelligence. I would guess that if you gave robots an R2D2 like port to jack into and play a game, that problem could be solved relatively quickly.
They also claimed it "learned" to play by playing itself only however it was clear that most of the advanced techniques were borrowed from existing AI and by observing humans.
No surprise they gave up on that project completely and I doubt they'll ever engage in anything like that again.
Money better spent on different marketing platforms.
Saying you've solved Dota after stripping out nearly all of its complexity is like saying you've solved Chess, but on a version where the back row is all Bishops.
Look at how long Theranos went on! Miraculous product. Attractive young founder with all the right pedigree, credentials, and contacts, dressed in black trurtlenecks. Hell, she even talked like Steve Jobs! Investors never had a chance.
That is what investors see. You seem to treat this as a purity contest where you define purity
As an ex dota player, I don't think this is that far off from having full on, all heroes dota. Certainly not as far of as you are making it sound.
And dota is one of the most complex games, I expect for example that an AI would instantly solve CS since aim is such a large part of the game.
It is certainly possible, but i won't be impressed by anything "playing CS" that isn't running a vision model on a display and moving a mouse, because that is the game. The game is not abstractly reacting to enemy positions and relocating the cursor, it's looking at a screen, seeing where the baddy is and then using this interface (the mouse) to get the cursor there as quickly as possible.
It would be like letting an AI plot its position on the field and what action its taking during a football match and then saying "Look, The AI would have scored dozens of times in this simulation, it is the greatest soccer player in the world!" No, sorry, the game actually requires you to locomote, abstractly describing your position may be fun but it's not the game
But then again, that is precisely the point. A chess bot also has access to gigabytes of perfect working memory. I don't see people complaining about that. It's perfectly valid to judge the best an AI can do vs the best a human can do. It's not really fair to take away exactly what a computer is good at from an AI and then say: "Look but the AI is now worse". Else you would also have to do it the other way around. How well could a human play dota if it only had access to the bot API. I don't think they would do well at all.
There are ~86 billion neurons in the human brain. If we assume each neuron stores a single bit a human also has access to gigabytes of working memory. If we assume each synapse is a bit that's terabytes. Petabytes is not unreasonable assuming 1kb of storage per synapse. (And more than 1kb is also not unreasonable.)
The whole point of the exercise is figuring out how much memory compares to a human brain.
I wouldn't have expected for anyone to even attempt to argue a human can beat, or even approach, a computer on working memory. Wikipedia is just 24.05gb. You are somehow claiming here that a human can hold that in working memory. That is they read it once and have perfect recall. Not even the most extreme savants have shown such feats.
We don't know what this means. Each neuron connects to thousands of synapses. I would assume that there is some quantity of information encoded in each pairwise connection of synapse-paths through a neuron. I would assume this is more than a bit, and also that it has something that might be described as a lossy fractal compression with each pathway adding or subtracting from whatever structures store information so that each path can use the same physical things - though not with perfect fidelity.
But the nuts and bolts are somewhat beside the point. The point is that if you look at Leela zero it only needs like 3GB of RAM to run and we have no evidence this is more memory than human grandmasters use to play chess. Yes, humans have imperfect recall but that's not relevant because neural net based chess engines do not work based on perfect recall.
And you can try to play some game where you create disadvantages to try to balance out all of the advantages of the machine interface, but again, hard to reason about the edge cases, and easy to create a misleading headline like "artificially limited reaction time worse than professional gamers" while in practice being able to react to information much more quickly than a human player because of its exclusive interface to game state. All of that is fair and well, and doesn't take anything away from the very cool achievements of google et al., but when you change the core mechanics of the game to accommodate a uniquely challenged player, you're playing a different game! Chess is ~mostly not about physically moving the pieces on the board, but Counter Strike is about little more than that! (And dota is somewhere in between.)
Only the first time, later when it played better players it always lost. Players learned the faults of the AI after some time in game and the AI had very bad late game so they always won later.
As we're learning with LLMs, the dataset is what matters - and what's awesome is that you can see that in us, as well! I've read that our evolution is comparatively slow to the rate of knowledge accumulation in the information age - and that what this means is that you can essentially take a caveman, raise them in our modern environment and they'll be just as intelligent as the average human today.
But the core of our intelligence is logic/problem solving. We just have to solve higher order problems today, like figuring out how to make that chart in excel do the thing you want, but in days past it was figuring out how to keep the fire lit when it's raining. When you look at it, we've possessed the very core of that problem solving ability for quite a while now. I think that is the key to why we are human, and our close ancestors monkeys are...still just monkeys.
It's that problem solving ability that we need to figure out how to produce within ML models, then we'll be cooking with gas!
AI clearly isn't at human level and it's OK to admit it.
Elon's response to this is that if we want these androids to replace human jobs then the lowest friction alternative is for the android to be able to do anything a human can do in a human amount of space. A specialized machine is faster and more efficient, but comes with engineering and integration costs that create a barrier to entry. Elon learned this lesson the hard way when he was building out the gigafactories and ended up having to hire a lot of people to do the work while they sorted out the issues with the robots. To someone like Elon a payroll is an ever growing parasite on a companies bottom line, far better if the entire thing is automated.