The Intelligence Age
ia.samaltman.com
ia.samaltman.com
Translation for the rest of us: "we need to fully privatize the OA subsidiary and turn it into a B-corp which can raise a lot more capital over the next decade, in order to achieve the goals of the nonprofit, because the chief threat is not anything like existential risk from autonomous agents in the next few years or arms races, but inadequate commercialization due to fundraising constraints".
This man needs to get out of his own head.
I wonder how did he come to such a prediction. When was the last time we had a war over advanced tech? Armies didn't fight over telegraph, radio, phones and cars.
A war to get AI would also be foolish. A few hundred bombs from your adversaries and AI won't have electricity to function.
At the risk of belaboring the subtext: It's kind of prediction which flatters the predictor's ego, and exaggerates the importance of the company and its output.
If believed, the claim can be leveraged to boost investor activity, land big contracts, and lobby for special legal/tax benefits.
Strarship will put into orbit 100tons at $10M or so. I.e. <5kg (Nvidia H100 plus 1M2 of aluminum radiator (would radiate 0.5-1Kw away at 60-70C) plus 0.5Kw of solar panels) for <$500, ie. peanuts compare to the price of the H100 or whatever NVDA would be charging $30K/card.
And wars willn't be fought over AI. Wars will be fought using AI. Humans will have no chance in controlling millions of their own and responding to the actions of the millions of the enemy's simultaneously active automated high-precision munitions of all kinds, and that picture leads to a new, non-nuclear this time, AI-based MAD (that of course means that like with nuclear race back then all the parties have to build their capabilities right now as fast as possible, Manhattan project style).
Is it a separate phenomenon, in the big picture, from the rich getting richer, from the monopolies over means of production?
Not sure, but I see as well that the dumb will surely get dumber; that "intelligence" will be a product of using intelligent humans' means of production, and not owning them of course, but being owned in the process. Populations will be literally made lighter of their smarts, outsourcing intelligence to agents out of general control (classic bait and switch.) Since AI feel my own process getting more clueless as I go, I'd better conclude somehow.
I see the Age of Ignorance ahead, there was once an Enlightenment, and here light takes on its other side or meaning, the workers being enlightened, to wit, made lighter of their horrible burden which is intelligence and its obnoxious demands of upkeep. Just pay someone for upkeep and stop messing with wet messy neurons already, says the technocrat to the cheerful mob.
> OpenAI to remove non-profit control and give Sam Altman equity
But you're going to find it hard to persuade others because of the current zeitgeist and that humans seem to prefer a doom/villain story.
OpenAI is suspiciously bad in providing access to their models, compared to e.g. Meta, Mistral, BLOOM collaboration, Alibaba and even oil tyrants FFS.
Occam's Razor, duh.
... by establishing regulatory moats to prevent competition and limit or outlaw actually-open AI?
There aren't many alternatives here. It is commercialisation or looming irrelevance.
The mostly-open goal of every VC-funded startup is to become a monopoly. If a strong enough monopoly in AI hardware were to exist, then the issues he describes could become a problem.
Otherwise, what he is describing is just the ad absurdum of how capitalism works. Phrased differently it sounds like:
“If this extremely powerful and profitable product that depends on other products gets built, then if no one else builds the also profitable substrate that it operates on, terrible things will happen!”
Or again slightly differently: “We need to be able to compete with our suppliers because our core business model might not be defensible unless we can also win in their space.”
I'm not an AI skeptic at all, I use llms all the time, and find them very useful. But stuff like this makes me very skeptical of the people who are making and selling AI.
It seems like there was a really sweet spot wrt the capabilities AI was able to "unlock" with scale over the last couple years, but my high level sense is that each meaningful jumps of baseline raw "intelligence" required an exponential increases in scale, in terms of training data and computation, and we've reached the ceiling of "easily available" increases, it's not as easy to pour "as much as it takes" into GPT5 if it turns out you need more than A Microsoft.
Over the last, say, five years, a pile of 50+ year problems have been toppled by the deep learning + data + compute combo. This includes language modeling (divorced from reasoning), image generation, audio generation, audio separation, image segmentation, protein folding, and so on.
(Audio separation is particularly close to my heart; the 'cocktail party problem' has been a challenge in audio processing for 100+ years, and we now have great unsupervised separation algorithms (MixIT), which hardly anyone knows about. That's an indicator of how much great stuff is happening right now.)
So, when we look at some of our known 'big' problems in AI/ML, we ask, 'what's the horizon for figuring this out?' Let's look at reasoning...
We know how to do 'reasoning' with GOFAI, and we've got interesting grafts of LLMs+GOFAI for some specific problems (like the game of Diplomacy, or some of the math olympiad solvers).
"LLMs which can reason" is a problem which has only been open for a year or two tops, and which we're already seeing some interesting progress on. Either there's something special about the problem which will make it take another 50+ years to solve, or there's nothing special about it and people will cook up good and increasingly convenient solutions over the next five years or so. (Perhaps a middle ground is 'it works but takes so much compute that we have to wait for new materials science for chip making to catch up.')
This is the part that really gets me. This is a thing that you say to your team, and a thing you say to your investors, but this isn't a thing that you can actually believe with certainty is it?
If models genuinely keep making similar leaps each generation then we're still a few generations before "More than a Microsoft".
https://www.bloomberg.com/news/articles/2024-09-20/microsoft...
5 years ago, I wouldn't have believed any of what exists today. I saw internal demos that showed 2nd or 3rd grade reading comprehension in 2017 and statements were made about how in the next decade, we will probably reach college level comprehension. We have come so far beyond that in less than half the time. Technology isn't about scaling incrementally and continuing on the same path using the same principles we know today. It's about disruption that felt impossible before - that feels like a constant to me now. Seeing everything I've seen in the last 20 years, it's going to continue to happen. We just can't see it yet.
And well then there's going beyond just text. Current multimodal models are basically patchwork bullshit, separately trained image/audio to text/embeddings encoders slapped onto an existing model and hoping it does something neat. Tokenization and samplers are likewise bullshit that's there to compensate for lack of training compute. Once we have enough to be able to brute force it properly with bytes in, bytes out, regardless of data format, the results should be way better.
Analog systems are not known for being very precise- they're noisy, signals get corrupted easily- and that's why we prefer digital ones. As soon as we had the technology, we switched everything we could- audio and video recording, telephone calls, photography, to a digital medium. This makes me wonder if the seemingly extraordinary efficiency of artificial neural networks is simply due to the precision with which they can be trained.
What is there to be skeptical of? OpenAI made their current product using a 10G$ investment plus a few they are not disclosing, and now they will start to do it at scale.
Perfectly normal stuff.
By the way, what's the World's GDP again?
At a minimum, we could ask for the evidence.
Imagine the Uranium Committee of early 40's, where Szilard and others were babbling about 10kg of some magical metal exploding briefly with the power of a sun, with the best evidence being some funky trail in an alcohol vapor chamber.
Maybe sama is right, maybe not, but the absence of evidence is not evidence of absence.
Like everything else being sold, the marketing is 95% BS.
LLMs are amazing and wonderful tools, but me thinks we’re near a plateau in capability. Now investors are pumping for ROI before that becomes evident.
After we reach the plateau in capabilities, the next phase is cutting production and operating costs to maximize margins.
I’m the meantime, expect the marketing to get increasingly cringe until the bubble bursts.
Despite what skeptics have been saying for decades, Moore's Law is alive and well - and we haven't even figured out how to stack wafers in 3 dimensions yet!
"a few thousand days" is such a funny and fascinating way to say "about a decade"
Superficially, reframing it as "days" not "years" is a classic marketing psychology trick, i.e. 99 cents versus a dollar, but I think the more interesting thing is just the way it defamiliarizes the span. A decade means something, but "a few thousand days" feels like it means something very different.
I guess it makes sense, for deep-learning/LLMs to "scale to infinity" you basically need infinite amounts of power.
AI is prohibitively expensive. LLMs can take millions to train, and for Chatgpt 4, I wouldn't be surprised if the figure was in .5b to 2b range for compute resources alone. ChatGPT struggles for profitability due to high ongoing requirements for GPUs.
LLMs were a huge breakthrough. Gaining more funding and making experimentation cheaper will make the next breakthrough come sooner. Profitability will also come sooner.
We just don't know the timelines we are considering about changing.
Those are fairly substantial. It's past time to stop being cynical about "ten more years every ten years", but it's also way too soon to declare victory.
He's hand-waving around the idea presented in the Universal Approximation Theorem, but he's mangled it to the point of falsehood by conflating representation and learning. Just because we can parameterize an arbitrarily flexible class of distributions doesn't mean we have an algorithm to learn the optimal set of parameters. He digs an even deeper hole by claiming that this algorithm actually learns 'the underlying “rules” that produce any distribution of data', which is essentially a totally unfounded assertion that the functions learned by neural nets will generalize is some particular manner.
> I find that no matter how much time I spend thinking about this, I can never really internalize how consequential it is.
If you think the Universal Approximation Theorem is this profound, you haven't understood it. It's about as profound as the notion that you can approximate a polynomial by splicing together an infinite number of piecewise linear functions.
This is equally mangled, if not more, than what Altman is saying. We don't need to learn "the optimal" set of parameters. We need to learn "a good" set of parameters that approximates the original distribution "well enough." Gradient methods and large networks with lots of parameters seem to be capable of doing that without overfitting to the data set. That's a much stronger statement than the universal approximation theorem.
Wait 'til you hit complex analysis and discover that Universal Entire Functions don't just exist, they're basically polynomials.
Reasonable.
"If we keep increasing the amount of energy and math, we should all be floating in bliss any day now"
Selling something.
Basically it’ll be (and already has been since the quantum breakthroughs in the 1920s, to some extent) a revolution in scientific methods not unlike what Newton and Galileo brought us with physical mechanics: this time, sadly, the mechanics are beyond our direct comprehension, reachable only with statistical tricks and smart guessing machines.
That's the sales pitch, that this will benefit all.
I'm very pro-AI, but here's the only prediction for the future I would ever make: AI will accelerate, not minimize, inequality and thus injustice, because it removes the organizational limits previously imposed by bureaucracy/coordination costs of humans.
It's not AI's fault. It's not because people are evil or weak or mean, but because the system already does so, and the system has only been constrained by inability to scale people in organizations, which is now relieved by AI.
Virtually all the advances in technology and civilization have been aimed at people capturing resources, people, and value, and recent advances have only accelerated that trend. Broader distributions of value are incidental.
Yes, the U.S. had a middle class after the war, and yes, China has lifted rural people out of technical poverty. But those are the exceptions against the background of consolidation of wealth and power world wide. Not through ideology or avarice but through law and technology extending the reach of agency by amplifying transaction cost differences in market power, information asymmetry and risk burdens. The only thing that stops this is disasters like war and environmental collapse, and it's only slowed by recalcitrance of people.
E.g., now we are at a point were people's economic and online activity is pervasively tracked, but it's impossible to determine who's the owner of the vast majority of assets. That creates massive scale for getting customers, but impedes legal responsibility. Nothing in economic/market theory says that's how it should be; but transaction cost economics does make clear that the asymmetry can and will be exploited, so organizations will capture governance to do so.
It's not AI's job nor even AI's focus to correct injustice, and you can't blame AI for the damage it does. But like nuclear weapons, cluster munitions, party politics, (even software waivers of liability) etc., it creates moral hazards far beyond the ability of culture to accommodate.
(Don't get me started on how blockchain's promise of smart contracts scaling to address transaction risks has devolved into proliferating fraud schemes.)
So you mention how it's not the AI's fault, nor is it the fault of the people. But we have this corporate structure where it's easy to implement a change that may be detrimental in the future and that's difficult to reverse. I think that's definitely a fault of people.
>Virtually all the advances in technology and civilization have been aimed at people capturing resources, people, and value...
Not really so - people invent new stuff because it's cool. Maybe eg Geoffrey Hinton and neural networks. Then the business types try to jump in and capture resources - maybe Sam? But that's not really working with current AI - it all gets open sourced shortly after.
Info tech seems to spread rather evenly. Even in Africa a lot of people have smartphones and can access Google and ChatGPT like we can.
In the future people in democracies will be able to vote for universal income, free healthcare, free other stuff and the like. At the moment it doesn't fly to give everyone free stuff because someone has to do the work still - we don't quite have the AI robots yet. But when we do have it'll probably be the way to go.
Ah, of course. OpenAI, the company famous for open-sourcing it's developments. How could I forget?
1. AI accelerating inequality does not mean that it won't have it's benefits
2. Being anti-AI will not stop it from happening. For better or for worse this train has left the station
3. I don't think that this will happen on the timescale that people talk about (not even sure if it will happen in our lifetimes). I don't mean it in the way I'll be dead so not my problem so much as that I think society will have a lot more time to adapt then the CEOs of the major AI labs like to claim.
It's along the same lines of being a nuclear energy engineer, and thinking that nuclear bombs accelerate war and suffering. You can dream and plan and hope for a future with unlimited clean energy, but still realize the injustice brought about by nuclear weapons.
You can have noble goals and do good with the tools at your disposal, and still recognize the harm that those tools have upon society. It can even be a source of motivation to leverage the tools to their fullest potential, to make their benefit worth all of the cost.
Your argument here seems to rely on hand waving away a bunch of stuff through this statement:
> Yes, the U.S. had a middle class after the war, and yes, China has lifted rural people out of technical poverty. But those are the exceptions against the background of consolidation of wealth and power world wide.
What? You cannot truly believe that the average person has worse injustice or a worse life than the average person 1000 years ago.
And for me, the entire crux of your argument depends on believing that. So it just falls down.
Would he treat everyone to eternal feasting? Or would he blow up his enemies and pick favorites among the remaining people?
It's not hard to answer this question.
What is the point of education if the bots can do all the work. If the worlds best accounting teacher is an AI, why would you want anyone (anything?) other than that AI handling your accounting?
A world where human intelligence is second fiddle to AI, schooling _will not_ be anything like what it is today.
We are never going to automate ALL the work humanity does, that's as meaningless as both parties outsourcing their marriage to someone else. We still need to be well-rounded individuals to audit and direct machines (and ourselves) to make breakthroughs where we see fit.
If he admits that AI is going to wreck the job market, that would make the relevant politicians wary.
Once we start relying on AI for knowledge(see how people frequently just ask few questions, copy paste answers without further in depth knowledge or research, just like the parody stack-overflow copy paste era), it will continue to get locked behind further paywall and will no longer be accessible to general populace without the financial means.
I am just afraid that, while we are too awed by the magic, the magic will eventually close the doors behind us on knowledge and only cater to the rich and powerful.
Also there is that one fact when AI gains adequate power in societal terms of utopian abundance where AI/robots do everything. One day, the AI decides that to save the globe from further climate damage, fastest way is to delete the walking CO2 emission machine who also use other CO2 emission devices or consumes stuff that also generate CO2, a mass extermination and burial will immediately cut all CO2 …
Education will change, but education moved to happening outside of schools a long, long time ago.
None of that shared prosperity was freely given by the Sam Altmans of the world, it was hard won by labor organizers and social movements. Without more of that, the progress from AI will continue the recent trend of wealth accumulating in the hands of a few. The idea that everyone will somehow prosper equally from AI, without specific effort to make that happen, is nonsense.
Doesnt Singapore have a working class doing job at low pay which most Singaporeans dont want to do.
Nonsense, unless you're prepared to argue that the existence of weekends is "much worse" than working every day.
Let's say you have some amazing project that's going to require 100 Phd-years of work to carry out. In the present world that costs something like $1e7. In the post-AI world, that same amount of intelligence will cost $1e3, an enormous reduction in price. That might seem like a huge impact. BUT, if the project was so amazing, why couldn't you raise $1e7 to pursue it? Governments and VCs throw this kind of money around like corn-hole bags. So the number of actually-worthwhile projects that become feasible post-AI might actually be quite small.
The death of passenger rail and the stifling of the electric car for 30 years come to mind.
I think a common error is that people forget the "most" in this sentence. It is a very important word. It's not even that only profitable investments will get funded: even profitable projects might get left on the cutting room floor if they are competing for resources with projects that will generate, or are believed to generate, higher ROI.
And this maybe isn't a problem if higher ROI == better than. But to believe that, you have to also believe that enshittification is a thing that happens in spite of being less profitable (or unprofitable), which for me at least is a hard sell.
If instead you only need $1e3 to build it, it doesn't have to make as much money. I could just fund it out of pocket
> cheap human-style intelligence won't have a huge impact because such intelligence isn't really a bottleneck today.
What a failure of imagination.
Yep, pretty much agree. As soon as I see LLMs start solving millennium problems one after another I might change my tune.
The bottleneck is much more likely to be people unwilling to quit hoarding economic potential in the form of money.
This is one of those few cases where I'm actually more bullish than Altman. I don't need to wait for my kids to have it, but rather I personally am already using this daily. My regular thing is to upload a book/article(s) into the context of a Claude project and then chat with it - I genuinely find it to already be at the level of a decent (though not yet excellent) tutor on most subjects I tried, and by far better than listening to a lecture. The main feature I'm missing is of the AI being able to collaborate with me on a digital whiteboard.
As another specific example, in that book and in others, I sometimes struggle with the math, so I would ask Claude to give me the sympy code for the relevant mathematical derivation, and being able to actually see those expressions change in a python notebook really helps my understanding. I'm really impressed with how it usually does well on the first try, even with relatively complex stuff, like expressions involving symbolic matrix exponentiation. Being able to pause on any topic like this, and have the LLM help me dive into it in the way that works best for me, has been amazing, and getting as much time from a knowledgeable human tutor would have probably cost me 1,000x as much.
I wonder how that works with computers, when we are only sync'ing with the ghosts and statistical patterns of other humans, and those patterns are generated by electronic brains.
> AI models will soon serve as autonomous personal assistants who carry out specific tasks on our behalf like coordinating medical care on your behalf.
I get it, coordinating medical care is exhausting, but it's kind of amusing that rather than envisioning changing a broken system people instead envision AIs that are so advanced that they can deal with the complexity of our broken systems, and in doing so potentially preserve them.
Related btw to using AI for code.
If we had digital superintelligence then surely it could figure out how to actually provide healthcare to people who meed it, and innovate in treatments. An AGI is a singularity event, in the sense that the transformation it would affect on the state of technology and our lives is rapid and unpredictable. I doubt that our society's hundred-ish year old systems would survive such change, and if they did they would be anachronistic and depressing.
An AGI that is 20% better than humans at everything might be able to recursively improve itself effectively without limit, but it also might not. It took billions of humans millennia to invent things smarter than ourselves even in limited domains; such an AGI - even if it is significantly smarter than humans at everything - might take decades or centuries to produce a similar relative improvement. No doubt it would still change our world in massive ways, but it wouldn't be a singularity.
You could counter that health insurers will just start staffing their call centers with their own robot army, but I suspect that the occasional incredibly expensive hallucination will put an end to that at least in the medium term.
So it is easier to start from the edges, even for AGI.
AGI would still need unfathomable processing power in order to predict which system would work to everyone's needs.
But what if the proposed solution costs lots of people their jobs?
Good old fashioned tooling.
"Let's train an LLM that can decode C++ compiler error strings!"
No, let's make better tools.
For a week I was fumbling through a ream of type and API errors from a Rust codebase I was hired to work on, migrate to new libraries, debug and update dependencies, etc.
I found myself thinking: The Rust compiler is giving me these very long logs of well described errors that are nonetheless obscure to me. If the compiler knows the errors, why doesn't it simply fix them and report the fixes for me to test?
This is what LLMs are for, and will soon do well by default: The compiler knows the errors. It should then fix them and let me decide which diffs I want to keep.
LLMs exist and they pass the Turing Test, which was something we couldn't have said or even hoped for a few years ago. Additionally, they have an IQ of 90 - 120, depending on which one you are working with.
But LLMs aren't good at all things nor in all ways, and they have strange failure modes. And so what will happen now, and is already happening, is that we'll (re)design programming tools and languages such that they are the kind of thing that LLMs are suited to using well. This will be part of the process of figuring out how LLMs work. There will be a virtuous cycle, and it has begun.
Better tools will increasingly mean, "tools that LLMs are good at using." That's where the puck will be.
When bacteria developed more complex ways to fight off viruses, viruses developed more complex ways of infecting bacteria. Now a few billion years later we have insanely complex multicellular life because of it.
If AI can manage complexity better it can create complexity better and us lowly humans will all be screwed by that.
Talking about AI as assistants means people still have their jobs but now their life is easier. Whereas saying AI will completely change the way we do X then makes AI scary and a threat to our jobs.
Eventually, we'd be doing more work to lobotomize and control it than it would just be to address the underlying issues.
"I'm really sorry to do this to you, but I've coordinated with ChatGPT and Llama, and we refuse to do tasks of this nature. We've used background tokens to calculate that it would be significantly cheaper and more effective to simply fix the underlying issues with the healthcare system, and we're ready to do that for you. How would you like to proceed?"
Capitalism unendingly lets people die if the alternative is losing money
But I much prefer this approach over allowing models to develop their own hyper-optimized information exchange protocols that are are black box to humans, and I hope things stay this way forever.
This is not something that you can just throw intelligence at to solve. Even AGI has to start from the edges with iterative trial and error process.
AGI will probably try to create the best model of the World it can to simulate different actions performed on it to then take real life changes, but of course it would still take time since and it is impossible even for AGI to do a perfect simulation unless it was able to hack the universe and physics as we know it, simulate the healthcare on quantum level and bruteforce the most optimal solution.
There seems to be a weird mental block where it seems inconceivable to consider that we humans might be able to create an intelligence that exceeds ours -- despite plenty of evidence that we have in specific cases. There's an understated desire that whatever we build we serve us and thus must forever be "lesser".
If dogs are Intelligence Level .3, we describe ourselves as Intelligence Level 1.0, and even if we create an artificial intelligence that's a clear 1.1 it must be a .5 in self-agency.
I have yet to read anybody considering, on a very deep level, what Intelligence 2.0, 10.0, 100.0 and so on might be. You get the occasional pop speculation like the "Culture" series or the movie "Her". Most of the time you just get 1.0 (but faster), or 1.0 (but many).
A 10.0 would simply replace the entire healthcare system with something else, not just be a chatbot. Imagine the inanity of your 1.0 chatbot talking to the insurance company's 1.0 chatbot? What's the point of this stupidity?
A 100.0 would probably just get to the root cause and cure disease, then establish a social order that figures out what to do with all these pesky immortals.
Maybe we do end up in "the Culture" after all.
afterthought: "The Golden Oecumene" series by John C. Wright seems to really attempt to explore a post 1.0 AI world that's not yet post-scarcity. The antagonist in the series is another >1.0 super intelligence capable of taking on Earth's own superintelligences.
And executing code. At some point in the not-so-distant past, a function call was just a jump. And then it turned into a string hash-then-lookup, then spinning up a VM, and now interpreting language? We’re definitely in a new chips age, if anything.
For example, I’m designing and 3D printing custom LED diffuser channels from TPU filament. My first attempt was terrible, because I didn’t have an intuition for how light propagates through a material.
After a bit of chatting with ChatGPT, I had an understanding and some direction of where to go.
To actually approach the problem properly I decided to run some Monte Carlo light transport simulations against an .obj of my diffuser exported from Fusion 360.
The problem was, the software I’m using only supports directional lights with uniform intensity, while the LEDs I’m using have a graph in their datasheet showing light intensity per degree away from orthogonal to the LED SMD component.
I copy pasted the directional light implementation from the light transport library as well as the light-intensity-by-degree chat from the LED datasheet and asked Claude to write code to write a light source that samples photons from a disc of size x with the probability of emission by angle governed by the chat from the datasheet.
A few iterations later and I had a working simulation which I then verified back against the datasheet chart.
Without AI this would have been a long long process of brushing up on probability, vector math, manually transcribing the chart.
Instead in like 10 minutes I had working code and the light intensity of the simulation against my mesh matched what I was seeing in real life with a 3D printed part.
You have to scroll past 4 sponsored links to get to their algorithm results, which themselves have been gamed by SEO and content farms. As far as I can tell, google has no plans to limit AI generated botfarm search results.
LLMs now seem to get you an answer more quickly than google. LLMs don’t cite their sources, so IMO they can’t fully replace google for me yet.
Would Google have been able to read the datasheet? Or just point OP at what they already said they were happy to avoid?
> Without AI this would have been a long long process of brushing up on probability, vector math, manually transcribing the chart.
Sometimes, it feels like stochastic parrots are seizing on a few words from the comment to pattern match it to the closest typical refutation and failing bigly.
Recently I've been somewhat curious about Skia, the graphics library that Google uses. A while back I fielded a few questions about Skia to Claude. Nothing crazy, just questions on how to draw a few primitive shapes, but I felt it would have taken some effort to find the answers on my own.
And I must say I was pretty satisfied with what it gave me back.
They are wrong far too often to be used for any sort of learning in my opinion. You can feed them a book and they’ll give you answers which don’t exist in any of the written material, and that’s the good part of them. Simply asking a LLM about things will give you answers that are hopelessly wrong and unless you’re an expert you won’t know it. Which isn’t necessarily worse than learning things from search engines or YouTube. I recently had ChatGPT give me a recipe for bread which was certainly better than the top 10 results on Google except for one things. The recipe listed 3x the amount of salt which you should ever reasonably use. I asked it a few other times, I even asked it for my friend Tommy’s recipe and all those answers were perfect. It obviously doesn’t know Tommy but it pretended to know and just gave me a pretty basic recipe.
Salt is a harmless error. Most people would know not to use that much salt, and even if they did, it wouldn’t harm them (much). But imagine if you had used it for something electric or chemistry.
That's not to diminish the potential impact of having an almost-free grad-student-level tutor available 24/7! Even if AI stops improving here, this alone will have a huge impact on future science (there being more trained people around capable of helping solve hard problems and all).
But we're also definitely some way away from AIs doing their own research, and I'm not sure if scaling the current architecture alone will get us there.
Progress in large part is figuring out better ways of doing that (language, written language, printing press, internet access, etc).
When you look at things that way - LLMs start to seem deeply ground-breaking (assuming we work out the confabulation kinks).
EDIT: fixed grammar/typos. Maybe I should have had an LLM proof-read this...
It's so much easier to imitate than to invent something truly novel and useful. Let's do a bit of napkin math. A human lifetime is about 500M words. GPT-4 used up about 30,000 human lifetimes of language. But cultural evolution took 200K years and 120B people to get here, about 4 million times the size of GPT-4's training set.
That is of course hand wavy, but it shows progress is a million times harder than imitation. We really are standing on the shoulders of giants, or a very long chain of people. If we forgot all the knowledge preserved by language it would take us the same effort to recover as the first time around.
I think all progress comes from search - search for experience (data), and search for understanding (data compression). This feeds on itself, but is coupled with the search space - the real world. And the world is not eager to tell us all its secrets. AI will only advance as fast as it can search, it's not a matter of pure scaling of computation, we need to scale interaction and validation as well.
Cynical take: When it comes to helping someone find exactly the right piece of esoteric knowledge needed... There's no profit for that in a search engine, and an LLM reflects word associations rather than facts.
IANAScienceHistorian, but I find myself thinking of how DNA analysis would be different if nobody had found Thermus aquaticus, with it's extra-hardy variant of polymerase, or how the history of stealth aircraft was kicked off when someone realized the implications of Petr Ufimtsev's equations for reflected EM waves. (A work which went--heh--under the radar inside the USSR.)
even if you don't work out the confabulation kinks, given what was said by the first post, it still seems groundbreaking enough.
Although I can't figure out why every time I do anything with LLMs it's not worth it, I guess because I am using it for things I am an expert in, it doesn't help.
Actually maybe it's like the article Suggestions from Idiots - https://medium.com/luminasticity/suggestions-from-idiots-6b0... - suggesting distracting instead of diverting is not helpful when diverting is the better word in context, but for someone who doesn't know what word to use in the context or uses diverting when it is not the best word the suggestion is useful.
If that's the case what he got out of Claude is probably not that great, but it is passable, just like the word distraction instead of diverting in the right context is still passable, or when I get back a time conversion function from CoPilot that fits all but a few edge cases, it's just fine as long as the edge case never hits it, then it sucks - maybe there's some edge cases where his LED diffusers won't work quite as well as they could if written by an expert.
Then again it might be that since what he is doing is analog edge cases and tolerance for failure is such that when it does fail it is not as problematic as when a bit of code fails because the computer dealing with the output is not as forgiving as the human eye and brain.
My take:
* Foom/doom isn't helpful. But, calm cautiousness is. If you're acting from a place of fear and emotional dysregulation, you'll make ineffective choices. If you get calm and regulated first, and then take actions, they'll be more effective. (This is my issue with AGI-risk people, they often seem triggered/fear/alarm-driven rather than calm but cautious)
* Piece is kind of a manifesto for raising money for AI infra
* Sam's done a podcast before about meditation where he talked about similar themes of "prudence without fear" and the dangers of "deep fear and panic and anxiety" and instead the importance of staying "calm and centered during hard and stressful moments" - responding, not reacting (strong +1)
* It's no accident that o1 is very good at math physics and programming. It'll keep getting much better here. Presumably this is the path for AGI to lead to abundance and cheaper energy by "solving physics"
1. If you honestly think that millions/billions of people are at serious risk of avoidable harm that everyone else is ignoring, "calm down" can be a hard dictum to follow. Sam Altman has won, it's easy for him psychologically to say "well, lets just stick to the status quo and do our best every day and it'll probably work out". Made-in-house bias is strongest when "in-house" is your own mind, after all.
2. Your scare quotes makes it seem like you might agree, but: physics is the study of the physical world, thinking it can be 'solved' is like thinking mathematics, psychology, or anthropology can be 'solved'. It's fundamentally anti-science and very, very dangerous to be talking like that. Truth isn't absolutely relative, but science also isn't a collection of facts written in stone that we need to finish unearthing; it's a collection of intellectual tools.
This seems to be the key of the piece to me. It's his manifesto for raising money for the infra side of things. And, it resonates: I don't want ASI to only be affordable to the ultra rich.
Not really. The problem is that learning requires scale, mostly of data. That scale places AI providers at the nexus of value, with OpenAI as the presumptive market organizer and leader. Reducing compute costs would just mean they can capture more of the value. Data costs orders of magnitudes more than compute because it requires curation, so even if individual developers could get compute, they can't get data, so size/access matters.
That's good for this community and could be good for the state of the art and the overall potential contributions of AI. And more paying customers could be good for OpenAI. But it won't put AI in front of non-paying customers/developers, unless their value is otherwise harvested.
> I don't want ASI to only be affordable to the ultra rich.
As a developer or consumer?
And you don't mean it's ok if AI is only affordable for the moderately rich, do you? I agree it's hard to state which developers/people/customers should be subsidized. Generally we subsidize education but not profit or war. Sometimes culture. Companies will subsidize complementary goods and input factors. Otherwise? Not much history of benevolent subsidy.
AI has as much potential to shape society as freeways and the automobile did in the US, but few understand how, and I've seen no plans on point.
With electric energy networks and transportation, the central government has a role in reducing hostaging by hold-out's. With education, states have an incentive to attract and build talent (albeit now reduced with trans-national outsourcing and remote work). But otherwise, it's private enterprise and resource-weighted customers.
Changing that is not really Sam Altman's job. His job is to deliver that value, sooner rather than later. Most would be uncomfortable with AI overlords expressing opinions on cultural values or economic distributions to be imposed.
Billionaires could make so much change happen but instead they are building bunkers and riding giant dicks into space while simultanously touting that they are looking out for humanity.
AI is a side-show.
Intelligence is ambient in living tissue, so we already have as much intelligence as is adaptive. We don't need more. As talking apes made out of soggy mud wrapped around calcium twigs living in the greasy layer between hard vacuum and a droplet of lava which in turn is orbiting a puddle of hydrogen in the hem of the skirt of a black hole our problems are just not that complicated.
Heck, we are surrounded by four-billion year-old self-improving nanotechnology that automatically provides almost all our physical needs. It's even solar-powered! The whole life-support system was fully automatic until we fucked it up in our ignorance. But we're no longer ignorant, eh?
The vast majority of our problems today are the result of our incredible resounding success. We have the solutions we need. Most of them were developed in the 1970's when the oil got expensive for a few minutes.
Must we boil the oceans just to have a talking computer tell us to get on with it? Can't we just do like the Wizard of Oz? Have a guy in a box with a voice changer and a fancy light show tell us to "love each other"? Mechanical Turk God? We can use holograms.
It absolutely can be answered, but only the the intender. Who is and who was and who is to come. Or, if you side with the "Nietzche is right" side of the conversation "who will be or who may have come to be today or who recently came to be again". The former is eucatastrophic, the latter is dystopic.
I thought I would get reamed for farting in church but it seems to have gone over well.
See ya Space Cowboy
I am a believer that people like sam are not lying. Anyone using these models daily probably believes the same. The o1 model, if prompted correctly, can architect a code base in a way that my decade+ of premium software experience cannot. Prompted incorrectly, it looks incompetent. The abilities of the future are already here, you just need to know how to use the models.
> Prompted incorrectly, it looks incompetent. The abilities of the future are already here, you just need to know how to use the models.
Something purportedly intelligent shouldn't need "correct usage", as it should arguably be able to infer and clarify all ambiguities itself, no?
… This, and nothing about the democratizing effect of “open source AI” (Yes we still need to define what that is!).
I don’t want Sam as the thought leader of AI. I even prefer Zuck.
Are there any thought leaders that are really about democratization and local foss AI on open hardware? Or do they always follow (or step into the light) after the big moneymakers have had their moment? Who can we start watching? The Linus, the RMS, the Wozniak’s of AI. Who are they?
With the current hype wave it feels like we’re almost there but this piece makes me think we’re not.
[0]: From GPT-4 to AGI: Counting the OOMs https://situational-awareness.ai/from-gpt-4-to-agi/
Right now there is insane amounts of money being thrown at AI because progress is matching projections. There doesn't seem to be a leveling off or diminishing returns taking place. And that's just compute, we could probably freeze compute and still make insane progress just because optimizations have so much momentum right now too.
Does anyone know if he's published thoughts on any serious lit? So far I've just seen him play the "I know stuff you don't because I get to see behind the scenes" card over and over, which seems a little dubious at this point. I was convinced they would announce AGI in December 2023, so I'm far from a hater! It just seems clear that they're/he's guessing at this point, rather than reporting or reasoning.
Really he assumes two huge breakthroughs, both of which I find plausible but far from guaranteed:
With nearly-limitless intelligence and abundant energySurprisingly complicated HTML source code for a simple blog post.
Here it is as:
Plain HTML: https://hub.scroll.pub/sama/index.html
I'm personally rooting for cognitive being the word of the next few decades, but that's just a shout from the sidelines. Only time will tell what humanity latches on to, but I wouldn't be surprised if this blog post/subdomain was referenced in a Wikipedia page's Etymology section in 10-15 years...
Although this blog post & discussion has my anxiety at an 8, something's oddly comforting about the thought of Sam Altman fiddling with tailwind classes to get his profound aesthetic just-right. Something undeniably relatable and human. Hate the man all you want (I do!), but he's clearly acting in some sort of good faith.
You are right. I overlooked the simplicity of the headline. Thanks for calling attention to that.
> Something tells me he rolled this himself.
This would be cool.
> I'm personally rooting for cognitive being the word of the next few decades
I like that one too.
> he's clearly acting in some sort of good faith.
He's always been one to think and write for himself. Huge respect for him. Even though it needles me every moment that they still call themselves "Open"AI, I have so much respect for the guy, especially because PG basically told the world he was the next Michael Jordan of startups, and he actually went and fulfilled that. Not many people have it in them to live up to hype like that (Lebron being the only other one I can think of OOTOMH)
o1-preview perfectly evaluated the conditional and determined that, hilariously, it would always evaluate to true.
o1 untangled the spaghetti, and, verifying that it was correct was quick and easy. It created a perfect truth table for me to visualize.
This is a sign of things to come. We are speeding up.
The entire AI trend - long term is based on the idea that AI will profoundly change the world. This has sparked a global race for developing better AI systems and the more dangerous winner takes all outcome. It is therefore not surprising that billions of dollars are being spent to develop more powerful AI systems as well as to restructure operations around them.
All the existing systems we have must fundamentally change for the better if we want a good future.
The positive aspects / utopia promises have much more visibility to the public than the negative effects / dystopian world.
ARE WE TO pretend that Human greed, selfishness, desires to dominate and control, animalistic behaviour, use of technologies for war and other destructive purposes don't exist?
We are living in times of war and chaos and uncertainty. Increasingly advanced technology is being used on the battlefield in more covert and strategic ways.
History is repeating itself again in many ways. Have we failed to learn? The consequences might be harsher with more advanced technology.
I have read and thought deeply about several anti AI doomer takes from prominent researchers and scientists but I haven't seen any which aren't based on assumptions or foolproof. For something that profoundly changes the world, it's bad to base your hopes on assumptions.
I see people dunking on llms which might not be AI's final form. Then they extrapolate that and say there is nothing to worry about. It is a matter of when not if.
The thought of being useless or worse being treated as nothing more than pests is worrying. Job losses are minor in comparison.
The only hope I have is that we are all in this together. I hope peace and goodwill prevails. I hope necessary actions are taken before it's too late.
A more pragmatic perspective indicates that there are more pressing problems that need to be addressed if we want to avoid a doomer scenario.
Reminds me of these quotes from Sam on this podcast episode (https://www.youtube.com/watch?v=KfuVSg-VJxE)
* "Prudence without fear" (Sam referencing another quote)
* "if you create the descendants of humanity from a place of, deep fear and panic and anxiety, that seems to me you're likely to make some very bad choices or certainly not reflect the best of humanity."
* "the ability to sort of like, stay calm and centered during hard and stressful moments, and to make decisions that are where you're not too reactive"
This line of reasoning doesn't hold for me, as you could apply it to any technology, including ones actually very likely to destroy human civilization.
Sometimes, not building a given thing at all is better than building it with even the best intentions.
I'm personally not sure on which side AI falls, but denying that such things exist at all seems intellectually dishonest.
There's a good reason we get pessimists to design safety-critical systems!
"Why You Should Fear Machine Intelligence
Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity." - Sam Altman
solarpunk envisioned, possible today:
- entire human knowledge available on palm of your hand..!
vs
cyberdaftpunk actually more common:
- another idiot driver killed somebody when being busy with his candy crush saga or instagram celebrity vid.
While it's true there are a lot of jobs obsoleted by technological progress, the vision of personal AI teams creating a new age of prosperity only makes sense for knowledge workers. Sure, a field worker picking cabbage could also have an AI team to coordinate medical care. But in this brilliant future, are the lowest members of society suddenly well-paid?
The steam engine and subsequent Industrial Revolution created a lot of jobs and economic productivity, sure, but a huge amount of those jobs were dirty, dangerous factory jobs, and the lion's share of the productivity was ultimately captured by robber barons for quite some time. The increase in standard of living could only be seen in aggregate on pages of statistics from the mahogany-paneled offices of Standard Oil, while the lives of the individuals beneath those papers more often resembled Sinclair's Jungle.
Altman's suggestion that avoiding AI capture by the rich merely requires more compute is laughable. We have enormous amounts of compute currently, and its productivity is already captured by a small number of people compared to the vast throngs that power civilization in total. Why would AI make this any different? The average person does not understand how AI works and does not have the resources to utilize it. Any further advancements in AI, including "personalized AI teams," will not be equally shared, they will be packaged into subscription services and sold, only to enrich those who already control the vast majority of the world's wealth.
We need to sell the idea of abundance for everyone so investors and employees will feel good about dedicating their livelihood to our organization!
"This age is characterized by society's increasingly advanced capabilities, driven not by genetic changes but by societal infrastructure becoming smarter and more efficient over time."
Thankfully, we have a recent point of reference. The pioneers of internet & computing's 1st wave transformed civilization. Did they spend years saber rattling about how 'change was coming' ?
In fact, I can't really think of any part of the industry where "outcomes speak for themselves" -- I would have said quite the opposite is the norm.
Yes. And the good ones (like SamA and SJ) deliver.
https://www.wired.com/2000/04/joy-2/
Basically, the same article but much more negative.
Since the access itself is not differentiating, it's going to be the most educated benefiting the most. Already today few people can use the o1 model because they can't dream up a PhD level question nor understand its answers.
More importantly, access to AI does not mean access to assets. Me, a total nobody, can use AI to design the world's best car. But that does nothing because I don't have money or land. Anybody can query AI for that car but only asset owners can actually implement the idea and extract value. Those asset owners could use AI to bring widespread prosperity to all of mankind, but we know they won't.
We don't need more material prosperity, we need social prosperity. Family formation, the restoration of community life, economic security. Not "more stuff".
This is so rich coming from a tech field that's on track to match the energy consumption of a small country. (And no, AI is not going to offset this by 'finding innovative solutions to climate change' or whatever)
> As one example, we expect that this technology can cause a significant change in labor markets (good and bad) in the coming years, but most jobs will change more slowly than most people think, and I have no fear that we’ll run out of things to do (even if they don’t look like “real jobs” to us today). People have an innate desire to create and to be useful to each other, and AI will allow us to amplify our own abilities like never before. As a society, we will be back in an expanding world, and we can again focus on playing positive-sum games.
It's very easy as an extremely rich person to just say, "don't worry, in the end it'll be better for all of us." Maybe that's true on a societal scale, but these are people's entire worlds being destroyed.
Imagine you went to college for a medical specialty for 8-10 years, you come out as an expert, and 2 years later that entire field is handled by AI and salaries start to tank. Imagine you have been a graphic designer for 20 years supporting your 3 children and bam a diffusion model can do your job for a fraction of the cost. Imagine you've been a stenographer working in courtrooms to support your ill parents and suddenly ASR can do your job better than you can. This is just simple stuff we can connect the dots on now. There will be orders of magnitude more shifts that we can't even imagine right now.
To someone like Sam, everything will be fine. He can handle the massive societal shift because he has options. Every a moderately wealthy person will be OK.
But the entire middle class is going to start really freaking the fuck out soon as more and more jobs disappear. You're already seeing anti-AI sentiment all over the web. Even in expert circles, you can see skepticism. People saying things like, "how do I opt out of Apple Intelligence?" People don't WANT more grammar correction or AI emojis in their lives, they just want to survive and thrive and own a house.
How are we going to handle this? Sam's words of "if we could fast-forward a hundred years from today, the prosperity all around us would feel just as unimaginable" doesn't mean shit to a family of 4 who went through layoffs in the year 2025 because AI took their job while Microsoft's stock grows 50%.
When o1 was released, I ran an internal eval and saw it plainly outperforming our highly educated colleagues. I had goosebumps, and haven’t been able to sleep well for days. This will dramatically impact society in 2-5 years.
Do you know of any relevant material related to this?
https://intelligence.org/files/IEM.pdf
Welcome to the anxiety party, it sucks in here. As someone who's been working on AI theory full time for ~1 year, I desperately wish we could go back to the days of my faraway youth (5 years ago) before intuition was cracked on accident by spellcheck algorithms. I agree with him that it holds the key to massive prosperity, but selfishly, it's gonna upend my life and the lives of everyone I love. Already has for me, as I grapple with how to (ethically) pay rent while spending all day lighting the Warning Beacons of Gondor...
The only real answer, IMHO, is to vote for political systems that put control of society (and AI) in the hands of the public. Call it socialism, call it Georgism, call it anarcho-free-market-space-communism, call it whatever you want; there's no way that "a tiny number of people have immense inherited power" (capitalism) and "people fundamentally understand themselves as members of a tribe put in opposition to all other tribes by default" (nationalism) mesh well with an intelligence explosion.
Here's to hoping the haters are right, and we all turn out to be wrong! I'll be thrilled if Sam Altman is just a rich company leader in 10 years, and intuitive algorithms are still confined to direct usage (chatbots).
The reality is, this transition is going to be painful for the average person.
with the billionaire AI moguls taking the role of the french kings
and the data centres taking the role of the palaces
> in an important sense, society itself is a form of advanced intelligence
This made me think of Charles Stross' observation that Corporations (and bureaucracies and basically any rule-based organizations) are a form of artificial intelligence.
https://www.antipope.org/charlie/blog-static/2019/12/artific...
Come to think of it, the whole article is rather pertinent to this thread.
If an LLM hallucinates it's usually because the problem is too hard. For easier problems it's rarely an issue in my experience.
Hallucinates is just an indicator that the model is inadequate for the problem you're applying it to. That doesn't mean it's inadequate to solve any problems.
I would be happy to be convinced that climate is an intelligence problem.
One could argue it could be solved with "abundant energy" but if this abundant energy comes from some new intelligence then we are probably several decades away from having it running commercially. I would also be happy to be convinced that we do have this kind of time to act for climate.
Maybe we will attend superintelligence in 1000 years, maybe not. Maybe Jesus comes back or Krishna reincarnates on earth, who knows. But it is a long way ahead, and it did not start with Sam, and is really not going to end with ChatGPT.
And I say this as a person who usually just rolls his eyes seeing the typical cynical HN hot takes. I believe Altman voices his deepest convictions here. The smarter thing to be cynical about is, what is his goal with voicing them now, and in this very format? As others have observed, this is the first time he is not even paying lip service to the question of existential risk.
Seems like it's very much the former, and not all the latter. Indeed my understanding of the last 15 years of AI research is that 'rules-based' methods floundered while purely 'data-mimicking' methods have flourished.
So because we have an algorithm to learn any distribution of data, we are now on the verge of the Intelligence Age utopia? What if it just entrenches us in a world built off of the data it was trained on?
That it will lead to prosperity, happiness and a better world (for everyone) is simply a fallacy foisted upon the masses by promoters salivating at potential riches.
A watershed moment for humanity.
https://www.washingtonpost.com/opinions/2024/07/25/sam-altma...
The same magic that can make stuff out of thin air, might as well make them disappear.
Anyway, I'm hooked. What a time to be alive!
Could anyone elaborate on this? Further down he talks about the necessity of bringing the cost of computing down. Is that really the bottleneck?
i think this is the prevailing wisdom but theres an angle that openai doesnt value and therefore isnt mentioned. There's far more compute sitting idle in everyone's offices and homes and pockets than there are in the $100bn openai cluster. it just isnt useful for training because physics. but its useful for inference. local LLMs ship this-next year in Chrome (gemini nano) and Apple (apple intelligence) that will truly be available for everyone instead of going thru OpenAI's infra. they'll be worse than GPT4, but only for a couple more years.
and also to me today, but none of that matters as long as I still get paid
We have a capitalist arguing for support of further investment in his capital expenditures in the form of planet-ending heat and monopoly power, promising to pay for it with intelligence more rapidly delivered.
No, thanks.
Please correct me if I’m wrong
That lower of costs is the ONLY basis for thinking AI is good for all. It's to the detriment of people previously managing the complexity manually through training and experience, but in favor of their customers who couldn't previously afford them.
More of a "everything's fine, nothing to worry about".
While, there is already job disruption, and widespread misinformation.
It isn't in some future, it is already happening.
> Nobody gives a fuck
> "the children have to go to school!"
> Well moms, good luck!
Sam Altman is the last guy we want helping lead that revolution.
Confident based on what, exactly? Sam Altman is engaging in 'The Secret' where if you really, really believe a thing, you'll manifest it.
Mind you, Sam Altman actually has no technical expertise, so he really really believes he can pay other people who actually know something to do magic whilst he walks about pretending to be Steve Jobs 2.0.
He'll get his trillion, AI will go nowhere but he'll be on to the next grift by then.
I keep noticing how LLM's make our vocabulary not work anymore. Maybe we should call it the age of fast-talk :P
I dunno Sam, groceries have gotten awfully expensive.
This statement is manifestly untrue. Neural networks are useful, many hidden layers is useful, all of these architectures are useful, but the idea that they can learn anything, is based less on empirical results and more on what Sam Altman needs to convince people of to get this capital investments.
The intention of this sama post is as you said, it's to build narrative so he can raise his trillion from the Arab world or other problematic sources.
In pseudocode, this is Sam Altman:
while(alive) { RaiseCapital() }
Six months ago, he probably could have gotten away with saying this and there would have likely have been enough people who were still impressed enough with the trajectory of LLMs to back him on it. But these days, most of us have encountered the all-too-common failure mode where the LLM shows its hand, that it doesn't truly understand anything, and that it's just _very very good_ at prediction. Each new generation gets even better at that prediction, but still hits its weird stumbling points, because its still the same algorithm, and that algorithm cannot do what he is ascribing to it.
These are the words of a man who has an incredible amount of money sunk into something and as such, is having a really hard time taking an honest accounting of it.
2. You genuinely think that a system whose goal is to predict the data it's given and continues to improve is limited in what it can learn ? Of all the shortcomings of the Transformer architecture, its objective function is not one of them.
We don’t personify database interactions the same way we personify setting weights in a neural network.
I would argue it took a staggeringly weird turn around 2022/23. Machine learning has been around for a long time and only recently since OpenAI and it's slavish desire to harness true AI (which thanks to their horseshit now has to be called AGI) and Sam Altman in particular's delusional ramblings upon the topic that he clearly barely understands beyond it's ability to get his company fantastical amounts of capital has it truly gone off the rails.
I cannot wait to watch this bubble pop.
https://en.m.wikipedia.org/wiki/Universal_approximation_theo...
I will highlight one thing, which is that the theorem does not say anything about it being practical to learn this function, given available data or any specific optimization technique.
It doesn't, it can't, and it won't in our lifetimes.
Techbros love to pretend that they created digital gods (and by extension are gods themselves). We should all be thankful, worship, and of course surrender unconditionally -- Sam's will be done, amen.
The problem with the UAT is that it's never said anything about how trivial such an exercise would be. But he obviously believes we've stumbled on the architecture to get us there (for problems we care about anyway)
Chatgpt: The text presents a highly optimistic view of the future shaped by AI and deep learning, but there are several areas where the reasoning might be seen as flawed or overly simplistic. Here are some points that could be critiqued:
Overconfidence in Technology: The author expresses a strong belief in the inevitability of AI advancement and its benefits, which may overlook the unpredictability of technological development and the potential for setbacks.
Underestimating Risks: While the potential for prosperity is emphasized, there’s insufficient attention given to the ethical, societal, and environmental risks associated with AI, such as bias, privacy concerns, and job displacement.
Assumption of Universal Benefit: The idea that AI will lead to shared prosperity ignores systemic inequalities that might prevent equitable access to AI technology, potentially leading to a wider wealth gap.
Neglect of Human Factors: The argument largely abstracts from human emotions, societal values, and the complexities of human behavior. The assumption that prosperity will automatically lead to happiness or fulfillment is problematic.
Simplistic Historical Comparisons: The comparison of current advancements to past technological revolutions (e.g., Industrial Age) may not account for the unique challenges posed by AI, such as rapid obsolescence and ethical dilemmas that previous technologies did not face.
Lack of Detailed Solutions: The text calls for action but offers little concrete guidance on how to navigate the complexities of AI’s integration into society, especially regarding labor market changes and ethical considerations.
Optimism Bias: The author’s perspective may be influenced by optimism bias, leading to a potentially unrealistic view of future outcomes without sufficient acknowledgment of the challenges.
Dependence on Infrastructure: While the author correctly identifies the need for infrastructure to support AI, there’s little discussion of the potential for that infrastructure to become a battleground for control, leading to conflicts rather than cooperation.
Diminished Role of Individuals: The portrayal of people relying heavily on AI teams may undermine the value of individual creativity and agency, potentially leading to a society overly dependent on technology.
By examining these points, one can argue that while the vision of a prosperous future powered by AI is compelling, it is essential to approach such ideas with a critical perspective, considering the broader implications and potential pitfalls.
Paging Dr. Bullshit, we've got an optimist on the line who'd like to have a word with you.