On being wrong about AI
scottaaronson.blog
scottaaronson.blog
We often get excited about these things, but society usually doesn't. That may be one way in which the current AI boom is different.
On the other hand, you can scrape the internet, build a model, semi-rig a couple of demos, hype the product and earn money.
Don't forget, it's all about money. Product is just a means to get money.
Building good things and earning good money as a byproduct is a thing of the old. Boomer stuff. Modern people don't do that anymore. It's not efficient, capitalist and quick enough.
I think that's a much more likely outcome than AGI.
Even if AGI is possible it's going to be constrained by the training resources thrown at it. Since most of those have been thoroughly enshittified now, the outcome for intelligent AGI isn't looking good.
Even if AGI hacks into everyone's camera and messages it's still going to turn into a spam+selfie moster superbot, and not Operation Paperclip 2.0.
This one in particular, I find fascinating, because we've literally just _had_ a hype cycle over it (I remember people telling me they'd be here any day now a decade ago), and now we're onto another AI hype cycle, almost immediately. Which comes first, AGI, or the self-driving car? :)
(I think your other examples are qualitatively different; moon bases, supersonic (passenger) jets and asteroid mining are clearly possible, but impractical for economic reasons; no-one is willing to _pay_ for them. Plenty of people are willing to pay for both self-driving cars and AGI, but there is just no clear route to achieving either.)
https://youtu.be/-Rxvl3INKSg is a news reporter's video of taking a ride in one, so you can see what it's like to ride in one. She's a news reporter and thus not your average person because she has access to Waymo because she's a reporter, but members of the general public are also getting access and are able to do this as well. After watching that, I think a reasonable average person could believe that self driving cars are here.
As was said upthread, the future is here but unevenly distributed.
There is no shortage of human intelligence in the world. The average person is remarkably intelligent. But they need a lot of learning and practice, and often plenty of organizational support, before they can use that intelligence to do something useful reliably.
We know how to make cars that are stupid and reliable. We know how to make cars that are intelligent and unreliable. But there is still a lot of work to be done before we can make cars that are both intelligent and reliable.
The only AI product I use in my daily work is Copilot. It can usually make useful and accurate suggestions, sometimes even remarkably so, but it also makes stupid mistakes all the time. Copilot is most useful for automating boring routine work, where I can quickly catch and correct its mistakes. It's far less useful for the difficult parts, or when I don't know what I'm doing, because the mistakes it makes are less obvious to me.
In the latent spaces of Stable Diffusion and friends.
First three are purely hardware problems, among other things (like profitability). But asking where are the self driving cars in 2023 is a bit disingenuous. Yes, we don't have Level 5 autonomy accessible by everyone for the general public, but we already have Waymo and other cars already driving on public streets, and we do have Tesla with their FSD that some people are using for 90+% of their daily commutes. So yeah, it took longer then some people anticipated (and will still take a bit longer until these systems are widely deployed, bulletproof and affordable), but we're definitely getting there.
We will continue integrating ai in all products for the next year's to come.
The money invested and what it opens up is still huge and it pointed a lot of people to this area for very good reasons.
But after the hype the work is there and now takes time.
Copilot rollout in my company (100k people) is only happening right now
Not at all, but maybe you didn't read far enough to get to it.
> The statement was from 2009. Back then, neither side had any real evidence in favour of their argument. Deep learning and AI were still purely academic experiments, with zero practicality.
That's exactly what the article says:
I was explaining that there was no trend you could knowably, reliably project into the future such that you’d end up with human-level AI. And in a sense, I was right.
and then it explicitly states the point:
The trouble, with hindsight, was that I placed the burden of proof only on those saying a dramatic change would happen, not on those saying it wouldn’t. Note that this is the exact same mistake most of the world made with COVID in early 2020.
I would sum up the lesson thus: one must never use radical ignorance as an excuse to default, in practice, to the guess that everything will stay basically the same.
Perhaps you didn't read my comment far enough to get it. The point is - essentially - you can't trust experts in any field when they start arguing beliefs instead of science. It has nothing to do with confidence in technological pacing or lack thereof. It seems the author has learned nothing of his real mistake and would be prone to repeating it in the future.
> The few who did predict what ended up happening, notably Ray Kurzweil, made lots of other confident predictions (e.g., the Singularity around 2045) that seemed so absurdly precise as to rule out the possibility that they were using any sound methodology.
The methodology (ignoring all other detailed knowledge) is roughly our progress with computers and software being exponential for its entire history. What lay people don't get about literal exponential growth[0] is how slow it is (vs linear, polynomial) before it 'hockey sticks'. I assume everyone here knows that and the Wikipedia link is only for the pictured graph as reference. If each unit on the x-axis is a decade of computer hardware and software advancements, we may very well be only two ticks away from the singularity.
https://lh6.googleusercontent.com/SPIkvg3D8tlFQvQJ8OEjFpTRdV...
And you can get out pen and project it into the future. The line is actually a bit faster increasing than exponential so curves up.
You can then compare that to human capabilities to get rough dates for stuff https://www.researchgate.net/figure/Kurzweils-8-71-chart-of-...
and that's pretty much all there is to it. It's held up quite well so far and has a good chance of doing so in the future.
It's not really that the increases were slow in the old days but that your old pc going faster didn't really mean much to people whereas overtaking human capabilities may be more of a thing.
Whether the trajectory will be civilization-altering is yet to be seen. Let me put it this way: I have serious doubts.
I disagree for your first two points. I do use generative AI to create image for my SWADE/DnD campaigns, but this is not art, this is illustration. There is no story behind the style, there is no story behind the color choice, or the meaning of the portraits.
AI was able to imitate Art since at least imagemagik. Kazimir Malevich's art, at least. Music is the same. Unless you think a DJ mixing up without mods or adding anything is an artist, in this case you might consider AI music art. I consider it entertainment.
Never got anything creative from ChatGPT, even when i was paying for it.
Bing chat (and copilot) is great to replace google, scaffold a lot of my code, and definitely boosted my productivity somewhat. But there's a reason why i don't use it to write my scenarii (still use it to spellcheck and syntax check), and even dialogs i have given up on (faster to prepare it myself in the end).
The only person I've seen talk sensible on these topics is Stuart Russell of "Human Compatible" (read the book, it's written with great clarity). He has his head firmly on his shoulders and eschews the breathless hype.
And this is just a few non-transformer machine learning approaches!
The closest I ever came to a "solution" is to target the economic forces which lead to people researching and innovating on AI. I eventually found fine-based bounties to be an unusually potent weapon, not only here, but against all kinds of possible grey- and black-urn technologies. I wrote up my thoughts very briefly about a year ago, and they still live at [url-redacted], but I suspect the argument is fundamentally flawed in a way I as a non-academic don't have time to suss out.
So I find myself in a strange place: Live my life mostly as normal, with a good chunk of my finances in low cost index funds, just in case the exponential starts shooting up - and just in case we don't all perish soon afterwards. C'est la vie.
Transformer architecture happened, that is key for neural networks to build up context dependent reasoning, we wouldn't have gotten to where we are today without it.
Without it scaling neural networks doesn't add much value, since without being able to separate out contexts they fail to develop a large amount of separate skills within the same model.
So yes, a very important key was found, it wasn't just scaling that got us here.
On which planet? Because i don't see any of it. AI should have made my console monkey job obsolete, by yesterday.
Also, same people learnt to get angry on anyone who points out that there's most probably a license violation happened during the process, and these angry people believe that everything is fair use (if you're powerful enough).
I mean, probably, sure, for some (generally poorly run) companies, but, when it comes to it, all 15 of the last two recessions have impacted hiring decisions. Companies, and particularly poorly-run companies, jump at shadows all the time.
Maybe that hype turns true, but if all AI development plateaued today and no new improvements were made to it then that AI's impact on the world would be very minimal. A useful tool for professionals, but not much more than that.
At least in the field of activity that I work in: No. I honestly don't have the slightest idea how AI could be really helpful for hiring decisions. The only "useful" usage of AI in hiring is to serve as a scapegoat for covering your a... if the hiring decision turned out to be a bad idea ("but the AI said ...").
It’s interesting so many people in the field thought neural networks would not lead to AI. Back when I started working with them around 2013, I thought we’d have AGI by 2030 (which was an especially quacky view back then that has now become an only slightly quacky view), but I also believed there was absolutely no way neural networks would be the approach that got us there. They seemed like fancy regression or curve interpolators—good perhaps for approximating molecular energies in an efficient way but not capable of having a conversation with me.
The “magic” that I thought neural networks were missing was algorithmic capability. Sure, they are universal function approximators, but that’s a bit of a hack theorem, and I didn’t think there was a realistic way to make use of that mathematical oddity; how could one practically train a neural network to compute a SHA256 hash? In fact, I still don’t think NNs can do that. But what I failed to realize, however, is that perhaps they could write the code to generate the hash. In retrospect, it seems kind of obvious, because of course the human brain can’t compute a hash function either—we just write code to do it as well.
Computer programs today seem to fall into one of two categories:
- “soft”: statistical learning or iterative linear algebra (NNs, SVMs, spectral methods, Monte Carlo, embeddings)
- “hard”: rigid, algorithmic, discontinuous (cryptography, mathematical proof systems, discrete optimization)
While most processes that occur in the human brain are probably characterizable as “soft”, I thought the uniqueness of human intelligence was due to a small but powerful amount of “hard” processing—and that we required a breakthrough in this area to achieve human-level AI. The release of GPT 3 immediately killed this viewpoint for me.
That said, while I now believe we may be able to achieve AGI using “soft” computation alone, I still think that achieving optimal AGI will require extremely “hard”, algorithmic computation. Optimal AGI would be that which performs better than any other computable algorithm on a very general problem space given some reasonable objective function. It’s quite possible there are many different starting points (non-optimal AIs) for getting there—neural networks being one of them—but as these systems recursively improve themselves, my guess is that they all end up converging on one universal, algorithmic, optimal AGI.
Pedestrians slight gestures, animals on the road, construction, police, wet cement, pot holes, fallen trees etc.
One needs to have somewhat a human experience and know the physics and behavior of almost all objects, including infering and learning on the fly unknown objects and environments.
Now if you build a system that does that, not only have you solved self driving cars, but robotics itself.
And if you solved robotics, that is inches away from AGI. Only a matter of time before it’s learned all physical trades, and only a matter of time a few humans use an army of robots to take over, or the robots themselves doing it in the goal of self preservation.
So inventing AGI is kind of conditional to L5 self driving and once you solve AGI, it’s a very different unpredictable world.
I fundamentally believe alignment is impossible. Sure you can align AGI to a few powerful humans but something aligned to all humans is very unlikely going to happen.
In the 1980s I was lucky enough to stumble into the opportunity of serving on a DARPA neural networks tools panel for a year and getting lucky applying a simple backprop model to a show-stopper problem for a bomb detector my company designed and built for the FAA. Bonus time!
Decades went by, and then deep learning really changed my view of AI and technology (I managed a deep learning team at Capital One, and it was a thrill to see DL applications there).
Now I am witnessing the same sort of transformations driven by attention based LLMs. It is interesting how differently people now view pros/cons/dangers/this-shit-will-save-the-world possibilities of AI. Short anecdote: last week my wife and I had friends for dinner. Our friends, a husband and wife team who have decades of success writing and producing content for movies and now streaming. I gave them a fairly deep demo of what ChatGPT Pro could do for fleshing out story ideas, generating images for movie posters and other content, etc. The husband was thrilled at the possibility of now being able to produce some of his previously tabled projects inexpensively. His wife had the opposite view, that this would cause many Hollywood jobs to be lost, and other valid worries. Normally our friends seem aligned in their views, but not in this instance.
I was describing myself as an observer and participant in different waves of AI tech. I was not admitting to being wrong, rather, just someone who was lucky enough to be a small part of AI.
What I don’t know right now: if AI will help solve difficult societal/energy/medical/scientific problems and propel humanity into a better future (I bet on this outcome) or that the AI-doomers are correct (not my view).
I try to absorb the AI dangers arguments and keep an open mind. Good resources: https://podcasts.apple.com/us/podcast/your-undivided-attenti... and I also think that privacy advocates (see books Surveillance Capitalism, and Privacy is Power) are useful in deciding what we should and shouldn’t do with AI.
My biggest anxiety with AI development rapidly increasing in pace and capabilities is that we don't work in a system which will distribute the efficiency gains brought by those developments to people, they will be captured as profit, we don't have anything in place on how to absorb large swaths of workers out of a job when an AI can help to automate 90% of most office jobs.
I really try to not be a Mennonite about it, technology invariably causes splash damage to jobs, from looms to robotic arms in factories, we increase our efficiency to produce but for some reason it feels like an AI boom will bring such a shift much quicker and broader than many past inventions. And I don't think we are prepared for the aftermath of that.
Let's say if in 10-20 years some 50% of white-collar jobs can be made redundant by AI, there would be large swaths of workers with non-marketable skills, how do you retrain so many people so quickly? We already struggle to retrain small pools of the labour force from jobs that are disappearing, like coal miners or factory workers, progress always come at a cost and so what's going to be the cost when that many people are out of jobs, out of prospect of jobs with their skills, and the value added by AI is captured by the few owners of capital?
That's my main anxiety. I have so far embraced AI as a way to make my job less tedious but I started to have this tingling feeling that I will see myself outskilled at my job by AI during my lifetime... And I have no idea what will be left to do. Given that predicting what will come next is nigh impossible, I don't think we will be prepared for the aftermath when it comes, just like we were not prepared for the aftermath of social media and just live with the malaise nowadays.
I view future AIs as being partners, working with people. That said, much less human labor will be required.
Not really addressing your good comments, but as an aside, I really hope that open AI models “win” rather than opaque AI systems owned and run by 3 or 4 giant tech companies.
I've came here to write a similar thing. Fifteen years is a long time, and just as you wrote, I also tend to change my perspective and opinions quite a bit (and sometimes even quite a lot) over much shorter time spans.
And I'm not even talking about tech--but about life in general. In fact, I remember some of my opinions from 5, 10, 15 year ago, and some of them are cringeworthy (to put it rather mildly) to my present self...
This is obviously true - predicting the future is hard. It doesn't however explain giving a timeline of thousands of years, which is a ridiculously long time for anything you believe to be technically feasible.
Turns out intelligence isn't complicated; it just needs a lot of brute force matrix math. This blog post can be interpreted as "I didn't realise what it was like living through an exponential process". Things happen very quickly.
[0] https://spectrum.ieee.org/estimate-human-brain-30-times-fast...
To see Aaronson say things like "2%" from within the walls of OpenAI is beyond sobering.
I'd say its more than just scaling, only the smallest forms run on a modern phone, slowly, with caveats.
Its far beyond anything reasonable. I still find it weird that there are probably multiple datacenters worth of compute just generating text.
In modern polarized environment you have to be hot or cold. In this example you either have to be AI doomer, or not. Nuances are not allowed, you will be voted down by both sides of argument.
People flock together in groups and sneer at one another with superiority.
It may have always been this way and I am just getting old.