John Carmack on AI
twitter.com
twitter.com
I fundamentally don’t like the crossbow and guns and second amendment argument. AI isn’t a tool to murder with, having no other function than the death of your neighbor. I feel like it’s more like the “should you be required to license your seeds from a megacorp” type thing. Food is existentially important, and it could be weaponized. But it’s not fundamentally a weapon, it’s fundamentally a thing of utility.
Also a billion or trillion or infinite ants can’t outsmart a single human. So unless some hard cap is enforced (how?) then mini AIs will probably not do anything
Cells are simpler than ants so can trillion or infinite cells outsmart a single human? Why not? How many cells is a single human?
The brain is about 0.1 trillion cells, of which 0.02 trillion cells are the cerebral cortex, which is occasionally consulted by the rest of the brain on matters such as what words to enter into this textarea.
Google can be accounted for? As in "I know the entity that has the data." or "can be held accountable" or "will be held accountable"?
You personally might not be able to exploit open AI fully, but there are lots of countries and orgs that could. Remember millions of dollars is loose change in the defense context.
AI is not powerful. AI doesn't increase information complexity or create new information. It actually does the opposite - it averages out the complexity peaks.
In effect, AI isn't intelligence, it dumbs everything down and reduces information to a common-denominator baseline.
(In the future the real difference between the "haves" and the "have nots" will be those who can understand and deal with information complexity and those who can't. Creating information complexity is the only thing humans are naturally good at, it's our biological imperative.)
I would still rate opportunity gap as higher priority over understanding complexity. its still funny when people try to be digital nomads just to discover geopolitics or time zones
I think neither elite actors like Sam Altman nor the peasants will manage to get their AI to do what they want it to do once the AI becomes knowledgeable enough and good enough at planning to make plans resistant to determined human opposition.
I.e., I think inequality is a distraction from the main problem with frontier AI research, namely, we are not going to figure out how to aim one such that it stays aimed after its cognitive capabilities have become dangerous.
https://chrome.google.com/webstore/detail/nitter-redirect/mo...
https://addons.mozilla.org/en-US/firefox/addon/nitter-redire...
The 'regulate it like nukes' thing because teenagers might ask it to end the world thing seems pretty far down the line.
I honestly don't give a shit about jobs, or having a job, I have one because the system says I need one, if I don't need a job, I'd be so so so much better off.
I'm certain we have enough technology and nohow to feed, house and provide electricity for everyone on earth without them having a 9-5.
How much can be up for debate, but giving it all to corporates is going to be a replay of the run up to WW2
I like working, I like making things, I just don't care about having a "job" which I tie my identity too.
I know that must jobs are bullshit jobs and the fact many need to show up to them just to provide for their family is stupid. Maybe a 3 hour day would be more realistic.
Also, I have a Japanese Toto I imported from Japan, it's literally a robotic toilet that wipes my ass for me. Try again angry man.
Yet that's exactly what the "leisure class" do. Inherit enough money and that's your life. Why is that not equally hilarious? Why do only some of us get to have that and it's OK?
You dont have it because you didnt earn it. Instead you're on the internet demanding like a child that you dont need to work for anything. Thats the difference between people with money and yourself.
And theres not a "pass" you or anyone else is entitled to give. Its not your property or family matter.
Bezos as an example built a great service. An exchange of value between willing customers and what he created happens repeatedly. Therefore he earned the billions. Thats how it works. The more value you create for the most people makes you more money. And I cant think of a more fair system.
Those Amazon servers don't magically teleport the packages to the customers.
All because he's lobbied, and continues to lobby, to not pay enough taxes. The tax system is unbelievably unfair.
Those who "create value" often don't. The continuing enshittification of everything is removing value from customers, yet the CEOs who are forcing it on their customers are getting paid vast salaries. We all know that they're destroying value and that their organisations will eventually be replaced by ones that actually value customers, but they continue to be rewarded.
You have a very naive view of capitalism.
You jest, but there's quite a lot of elderly, frail, incontinent boomers who are or soon will be in need of this service.
The Millennials zoomers and alphas seem to have little to no interest in doing the asswiping themselves.
Sounds a bit like Douglas Adams' Golgafrinchan Ark Fleet Ship B, except for tech bros instead of telephone sanitisers or advertising executives.
How would this work? Do you have to meet in person?
Theres a reason why Jamie Dimon is saying people will be working 3 days per week in a generation because of AI.
And its why in capitalistic and technology forward societies the poor are generally better off than every generation prior.
I have my reservations that THIS revolution will be the one that solves our problems. A jump in technology is simply a way for those with the means to get more out of those without. AI isn't an equalizer, compute seems like it'll be the new valuable resource and you can buy that with cash.
...
> Suppose that at a given moment a certain number of people are engaged in the manufacture of pins. They make as many pins as the world needs, working (say) eight hours a day. Someone makes an invention by which the same number of men can make twice as many pins as before. But the world does not need twice as many pins: pins are already so cheap that hardly any more will be bought at a lower price. In a sensible world everybody concerned in the manufacture of pins would take to working four hours instead of eight, and everything else would go on as before. But in the actual world this would be thought demoralizing. The men still work eight hours, there are too many pins, some employers go bankrupt, and half the men previously concerned in making pins are thrown out of work.
...
> If the ordinary wage-earner worked four hours a day there would be enough for everybody, and no unemployment — assuming a certain very moderate amount of sensible organization. This idea shocks the well-to-do, because they are convinced that the poor would not know how to use so much leisure. In America men often work long hours even when they are already well-off; such men, naturally, are indignant at the idea of leisure for wage-earners except as the grim punishment of unemployment, in fact, they dislike leisure even for their sons.
Bertrand Russell was already talking about how the eight hours workday was a wasteful bullshit in 1932.
The well-to-do also need to have an underclass to distinguish themselves from, otherwise they become common. To them, becoming common is one of the worst things that could happen.
Right now the main thing AI is good at is quickly generating bullshit, so those are the jobs that are going to be first to be replaced.
It's straight up absurd. AI is going to have no appreciable effect on this, because it would simply be disseminating existing knowledge. You can already look up how to build a nuclear bomb. It's not some kind of big secret that nobody's allowed to know. The same goes for all the other world-ending scenarios. AI can't act on its own in the physical world. Even when it eventually does, it will be bound by the same limitations people are.
I agree that the real dangers AI presents are much more mundane. People being able to do things so much more efficiently is going to cause instability in people's lives whose jobs it will affect. It won't replace everyone, but it will cause problems - people have to learn new skills, find new jobs etc.
If it can earn money or is given money, it can certainly get people to do things in the physical world.
Sure, it's a rather philosophical argument currently, but this will change.
If I would be a malicious actor, I'd create a chat bot that sneaks in responses that are in favor of my agenda. In that I'll make my users the puppets of the AI.
They're already doing this as there's a clear far left bias/lecturing that occurs in the models, especially those put out by OpenAI.
The AI doesnt care or know. its about what source the material is taught on. Turns out a lot literature, academic texts and even stuff on the internet is leaning left (and right). So this naturally seeps in.
When you live in a bubble left or right you will see the problem everywhere.
Can you elaborate more?
I guess the more interesting question is whether there are lasting changes. I'm not an expert in learning, memory, or brain development. Even without much knowledge it should be clear that frequent interaction will have lasting changes. For example, ChatGPT is biased in the sense that it does not know about all the books that haven't been digitalized. Used frequently, those missing books will be reflected in my brain.
Another example could be a chess training AI. If I train with an AI, I could have blind spots in the landscape of chess skills that are deliberately (or not) excluded. So clearly, this AI would change my brain but not only in a way I intend.
Asking the average Joe to crank out their own personal LLM is like asking the average Joe to stand up their own Mastodon (i.e. unrealistic).
When given the choice between owning your data and spending two minutes to do so vs. opening $MEGA_LLM in your browser and telling it to make you Instagram for Guinea Pigs, $MEGA_LLM will win every time. Doubly so now that $MEGA_LLM is starting to appear in _everything_.
Case in point: OpenAI is already well-past 100M users. many of them are _paid_ users (which is unprecedented).
I don't think running your own LLM should be illegal (under what grounds would that happen anyway?), but I'm also concerned about the world depending on, like, five huge entities for all knowledge.
yet
Having a single entity that tells you what it thinks is the right answer short circuits needing to think critically about the answer that was provided. This is especially bad when, in the case of ChatGPT, people are using it as a therapist, doctor, teacher, lawyer, pair programmer, and more.
There is also the possibility of a central authority using the platform to tell you how to think. When you control the model, you control the narrative. I think this could be extremely dangerous in the wrong hands.
The peasants deserve crossbows ;)
We have people calling for bans on code and math based on nothing more than sci-fi.
I too agree with Carmack that AI doomerism has a definite subtext of concern for changes in power dynamics.
what is that mean, anyways?
no:
"there isn't any"
real world evidence:
"reason to believe"
that they will become dangerous
i think he's referencing how the only reason anyone would think that there's any kind of existential threat is from sci-fi doomerism
however i feel like perhaps i missed the point of your comment, if so could you explain?
"There is no evidence that a black swan will be ever found". Hmm. OK.
There hasn't been evidence that AI doesn't pose an actual danger to human society. There are lots of predictions that it could not, or fictional scenarios where it does not. But no actual real-world cases.
See my point?
Banning things on the basis of evidence free speculation is a really dangerous road to go down.
For example, if you were being hired by law enforcement and you had written a number of books that stated that being a serial killer was a good thing, then you wouldn't get hired.
There appears to be a particular type of black and white response in this thread that seemingly ignores that the world is black and white.
Again - there is no evidence (and cannot be! - by definition) of things that never happened before.
Forecasts and opinions are not evidence, that's it.
Meanwhile AI dooms day scenarios use sci-fi movies as evidence and humanity has "survived" many technological revolutions in the past. Do you think everyone would care as much if the Terminator movies weren't so popular?
Personally I think there's some risk and problems to solve but it doesn't warrant the authoritarian panic responses it gets.
woah what and why
Besides, we have knives. I'm afraid this mindset will soon demand everything to be pre-cut in licensed facilities and make the knives illegal.
This desire for censorship and restrictions is infinitely ridiculous. Instead of addressing the issues of why would someone want to hurt someone or themselves, they try to eliminate the tools.
This is going to fail, maybe people in UK or in the USA might end up not knowing how things work without formal licensing but people in Russia, Ukraine, Bulgaria, China, Vietnam will know.
This attitude can actually lead to the downfall of the western societies. WTF the computer wouldn't tell me how to cook meth? That's something that junkies do and it's not the lack of knowing how to cook it that keeps me from doing it. I'm just curious, not looking to harm myself or anybody but I'm being lectured by a computer.
I would like to mention that in the Fallout series, a game that depicts a different version of history where nuclear power was widely accessible, cars and robots have their own nuclear generators. The reality is that if nuclear power was common, we would not have any problems with energy or a power grid.
It’s also not a secret how to do these things, they teach it to kids. The complexities come from the engineering the device and that’s actually the secret part.
Or am I mistaken and you could make it so that IEDs from nuclear energy couldn't be done?
Portugal is not much safer from IEDs than Irak because Portuguese are bad at chemistry and Iraqis are all heisenbergs.
I... don't see how that doesn't put a massive hazard on the availability of nuclear power generators everywhere? We have people willing to make and deploy IEDs today. Why would we not have them if nuclear sources were ubiquitous?
(If the idea is we would have the nuclear plants, but that the ubiquitous electronics would not have their own nuclear sources, I can see that. :) )
Personally, I'm not anti-nuclear but I'm also not for proliferation of it simply because I don't trust that there are enough serious people to operate a nuclear plant in every city at the same safety levels we have today. That said, I will never advocate for hiding the knowledge for building those.
as the old analogy goes, how big of a chunk of metal do you need to weld to the rail to kill everyone on the train? which part of that should be more illegal?
gun violence is a culture problem, not a security problem
know your threat model
But it's kinda futile, it's just the illusion of having power and self-governance, when the others have machine guns and worse.
I'm just saying... if the analogy you're using to justify your position seems to be entirely wrong doesn't that kind of undermine your position? It's not obvious to me that "peasants with crossbows" is helping here.
Younger me needed to hear that. Hopefully it saves someone else some time and energy.
Makes me think of the second amendment and John's stance on that; a "well-armed militia" might be a force to be reckoned with, but if it comes to an actual confrontation they're just peasants with crossbows vs the US military.
Not that I believe the US military would ever go up against their own people, mind you.
History is littered with armies losing to peasants. The US military lost to a bunch of poor people in the jungle then lost to a bunch of poor people in the desert.
US citizens, even just Texas, would most likely prevail easily over the US military especially as the soldiers would be wary of attacking their neighbors.
"What are your thoughts on gun control?"
with "Yes, I am pro 2nd amendment."
Like yes, of course, but are you implying carte blanch RPG's, Howitzers, and fighter jets for everyone?
Because if so, then I absolutely don't trust you to make sure AGI doesn't screw up the world.
And if not, then maybe you're not so "pro 2nd amendment" by interpretation.
Meanwhile, they're distracting the public away from the real issues, which are the same as always: filling the web with more garbage, getting people to buy stuff, decreasing attention spans and critical thinking.
The threat of job loss is also real, though we should admit that many of the writing jobs eliminated by LLMs are part of the same attention economy.
At least this is what I find more worrisome personally. But realistically I'm not sure we can keep it away from them, similar to how North Korea managed to acquire nukes, except it seems way easier to acquire powerful AI in this case.
It's like the culture wars again; there also seems quite a consistent mapping with the left being more on the safety/control side; the right reckoning it will all sort itself out if we let the market get on with it, contemptuous dismissals marking lines in the sand.
And it's only really just started. :(
Just like once you start reading greek philosophers, you realize all the important questions have been asked 2K years ago. Since then, we've only been trying to find the best ways to accommodate with our human nature.
Until GPU's became powerful enough, there really wasn't a way to do neural networks at scale, which was one of the reason it stagnated in the 80's.
One of the main reasons, GPU got powerful is because of pc gaming.
One of the main drivers of PC gaming was John Carmack and id.
I guess not, or we wouldn't know of him.
Until I existed to be part of the large market Carmack and the GPU makers were appealing to, Carmack wasn't working on PC gaming and id.
When the masses came, the results became gradually worse, but I still use the internet for answers instead of thinking on my own, but in many cases I'm forced to think on my own anyway.
With AI the source information is already polluted and the program is fantasizing its answers. But there will be a point where it delivers accurate answers.
My child, should I ever have one, would then use AI like any other tool. Will it have more resources at its disposal or would it be better that it comes up with its own answers?
I am a single child. I spent a lot of time alone, thinking, making sense of the world, figuring things out on my own. I played C64 games, with terrible graphics, which trained my imagination. I learned English by playing those computer games. There came a time where I outgrew those simplistic computer games, turned to having a social life, going to parties, getting into music creation, DJing.
Would that also be something my child would do? Use AI, grow up with values the AI suggests to it, abandon it later, search for its own way in life? Would it even learn another language, when the AI conveniently translates everything on the fly?
I think, before adding yet another thing on our plate most people, including me, don't fully understand, maybe we should focus on the energy problem and having a sustainable place to live.
Amy technological advance is based on having enough energy to fuel it, and in the state this planet, our habitat is right now because of years of uncontrolled energy use, energy must be a priority before anything else.
Unless you believe that AI can solve that riddle. Recently in the news Microsoft wanted to build small nuclear reactors to power its AI construct, adding a long term health and waste problem.
It's October, it's been over 25°C the recent days, the sea is still 21-23°C. Usually it's much colder. The amount of fires and floods this year is over the top.
Well sorry if that went all over the place.
Every generation has its "kids these days", but up until now they've all ended up all right.
(I say until now, because the current young generation is struggling economically because of the preceding generations messing up the system)
Carmack:
> To answer the question directly: there is no way, even with an infinitely smart (but not supernatural) oracle, that a teenager can destroy all of humanity in six months.
Even the most devastating disasters people imagine will not kill every human being even over the space of years. This is a common failure of imaginative scope, not understanding all the diverse situations that people are already in, and how they will respond to disasters.
You could walk it back to “kill a million people” and get into the realm of an actual discussion, but it isn’t one I am very interested in having.
---
I do think that's a discussion very worth having, asymmetrical advantage that AI gives is the concern here. A virus is a special arrangement of a few thousand atoms that we can now control and design on a computer. The defence is global vaccination programs costing billions of dollars and many lives in the process
We will no doubt be able to use AI to develop vaccines, that isn't the bottleneck, the bottleneck is in testing and distribution. It's a fundamentally asymmetrical situation
To be clear this is a risk without AI, but AI brings down the threshold of effort to make this more likely. I would love to be convinced otherwise
It quickly became apparent that Covid could mutate quicker than we could possibly keep up with. Like the common cold and the flu, it iterates too quickly to eliminate entirely like smallpox.
At any rate, it seems all you need for AI is attention (transformers + processing power). If AI ever becomes a real threat outside of generating believable fake images and videos, it's likely the counter is a diversity of AI models that can compete with the "evil one".
We will no doubt be able to use AI to develop vaccines that isn't the bottleneck, the bottleneck is in testing and distribution (and humans not wanting to get vaccinated). It's a fundamentally asymmetrical situation
Which was observed in real-time with the 'rona mutating to become more contagious, but less dangerous.
You don't need an AI to construct a virus that can destroy humanity. We already have that capability, and bioprinting one by average grad at random biotech on their lunch break will become commonplace soon.
Here's a youtuber designing a custom virus to cure his lactose intolerance https://www.youtube.com/watch?v=J3FcbFqSoQY&ab_channel=TheTh...
Bitcoin, Ron Paul, etc.
Carmack IMHO can best be described as an independent - some of his opinions will just so happen to align with one side or the other.