Google Brain founder says big tech is lying about AI danger
afr.com
afr.com
> [Ng] said that the “bad idea that AI could make us go extinct” was merging with the “bad idea that a good way to make AI safer is to impose burdensome licensing requirements” on the AI industry.
> “There’s a standard regulatory capture playbook that has played out in other industries, and I would hate to see that executed successfully in AI.”
> “Just to be clear, AI has caused harm. Self-driving cars have killed people. In 2010, an automated trading algorithm crashed the stock market. Regulation has a role. But just because regulation could be helpful doesn’t mean we want bad regulation.”
[1]: https://www.afr.com/technology/google-brain-founder-says-big...
[2]: https://web.archive.org/web/20231030062420/https://www.afr.c...
But imagine all the money bigco can make by crippling small startups from innovating and competing with them! It's for your own safety. Move along citizen.
The only meaningful thing in this discussion is about people who want to make money easy, but can’t, because of the rules they don’t like.
Well, suck it up.
You don’t get to make a cheap shity factory that pours its waste into the local river either.
Rules exist for a reason.
You want the life style but also all the good things and also no rules. You can’t have all the cake and eat it too.
/shrug
If China builds amazing AI tech (and they will) then the rest of the world will just use it. Some of it will be open source. It won’t be a big deal.
This “we must out compete China by being as shit and horrible as they are” meme is stupid.
If you want to live in China, go live in China. I assure you you will not find it to be the law less free hold of “anything goes” that you somehow imagine.
The trouble is sometimes they don't. Or they do exist for a reason but the rules are still absurd and net harmful because they're incompetently drafted. Or the real reason is bad and the rules are doing what they were intended to do but they were intended to do something bad.
> If China builds amazing AI tech (and they will) then the rest of the world will just use it.
Not if it's banned elsewhere, or they allow people to use it without publishing it, e.g. by offering it as a service.
And it matters a lot who controls something. "AI" potentially has a lot of power, even non-AGI AI -- it can create economic efficiency, or it can manipulate people. If an adversarial entity has greater economic efficiency, they can outcompete you -- the way the US won the Cold War was essentially by having a stronger economy. If an adversarial entity has a greater ability to manipulate people, that could be even worse.
> If you want to live in China, go live in China. I assure you you will not find it to be the law less free hold of “anything goes” that you somehow imagine.
But that's precisely the issue -- it's not an anarchy, it's an authoritarian competing nation state. We have to be better than them so the country that has an elected government and constitutional protections for human rights is the one with an economic advantage, because it isn't a law of nature that those things always go together, but it's a world-eating disaster if they don't.
When we ask it questions, an AI can tailor its answers to change peoples opinions and how people think. They would have the power to influence elections, our values, our sense of right and wrong.
That's before we start allowing AI to just start making purchasing decisions for us with little or no oversight.
The only answer I see is for us all to have our own AI's that we have trained, understand, and trust. For me this means it runs on my hardware and answers only to me. (And not locked behind regulation)
"Don't worry if our adversary develops nuclear weapons and we won't - it's OK we'll just use theirs"
The problem is what the Powers-That-Be say and what they do are not in alignment.
We are now, after much long-time pressure from everyone not in power saying that being friendly with China doesn't work, waging a cold war against China and presumably we want to win that cold war. On the other hand, we just keep giving silver platter after silver platter to China.
So do we want the coming of Pax Sino or do we still want Pax Americana?
If we defer to history, we are about due for another changing of the guard as empires generally do not last more than a few hundred years if that, and the west seems poised to make that prophecy self-fulfilling.
A price paid, I think, due a conformant, restrictive culture. And after all, even if you do excel, you may soon disappear.
It’s also a mistake to underestimate the market value of copies and clones. In many cases a cloned version of a product is better than the original. E.g., clones that remove over-engineering of the original and simplify the product down to its basic idea and offer it at a lower price.
It’s also a mistake to confuse manufacturing prowess for the ability to make “copies.” It’s not China’s fault that its competitors quite literally won’t bother producing in their own country.
It’s also a mistake to confuse a gain of experience for stealing intellectual property. A good deal of innovation in Silicon Valley comes from the fact that developers can move to new companies without non-compete clauses and take what they learned from their last job to build new, sophisticated software.
The fact that a bunch of Western companies set up factories in China and simultaneously expect Chinese employees and managers to gain zero experience and skill in that industry is incredibly contradictory. If we build a satellite office for Google and Apple in Austin, Texas then we shouldn’t be surprised that Austin, Texas becomes a hub for software startups, some of which compete with the companies that chose Austin in the first place.
FWIW There was a time when that was was the received wisdom about the USA, from the point of view of European powers. It was shortsighted, and not particularly accurate then either.
I once spoke to a Chinese person who speculated: "I wish that the Chinese were as conformant and uniform as the Americans - China is too diverse and unruly!"
I think that it's a common human habit to upsell one's own diversity and downplay that of others.
Explain DJI and Douyin/TikTok.
Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html
Imagine someone invents a machine that can give infinite energy.
Do you
a) sell that energy, or b) give the technology to build the machine to everyone.
Clearly b is better for society, a is locking up profits.
This non-hypothetical got us global warming already
Technically: a solar panel is just such a machine. You'll have to wait a long, long time but the degradation is slow enough that you can probably use a panel for more than several human life times at ever decreasing output. You will probably find it more economical to replace the panel at some point because of the amount of space it occupies and the fact that newer generations of solar panels will do that much better in that same space. But there isn't any hard technical reason why you should discard one after 10, 30 or 100 years. Of course 'infinite' would require the panel to be 'infinitely durable' and likely at some point it will suffer mechanical damage. But that's not a feature of the panel itself.
Requiring attribution categorically across the vast domain of generative AI...can you please elaborate?
i think it's a reasonable ask to enforce attribution of AI generated content. We enforce food labels, why not content?
I would go further and argue that AI generated content do not get granted the same copyright as human generated content, but with that, AI generated content using existing copyrighted training data does not violate copyright.
Regulation isn't always, but often is a drag on productivity. Food labels make total sense because the negative consequences of not doing it outweight the drag of doing it.
I'm not at all convinced that enforcing AI labeling and the resulting impossible task of policing and enforcing this will outweigh any negatives of not doing it.
I'm thinking about the cookie policy in Europe. I hate it and almost always just click through because so many websites work around it by making it a real pain to "reject cookies".
If you use an AI coding assistant then will the written code be without copyright? Or will the code require a disclaimer that says some parts of it are AI generated?
You're also going to have to be very precise on defining what AI means. For most people a compiler is as magical as AI. They might even consider it AI, especially if it does some kind of automatic performance optimizations - after all, that's not the behavior the user wrote.
This capability is clearly different from the examples you list.
Just because there may be no precise engineering definition does not mean that we cannot arrive at a suitable legal/political definition. The ability to create new content out of whole cloth is quite separate from filters, cropping, and generic "pre-AI" image post-processing. Ditto for spellcheck and word processors for text.
The line actually is pretty clear here.
not the most convincing of arguments
- it's actually to get regulatory capture
- it's hubris, they're trying to seem more important and powerful than they are
Both of these explanations strike me as too clever by half. I think the parsimonious explanation is that people are actually concerned about the dangers of AI. Maybe they're wrong, but I don't think this kind of incredulous conspiratorial reaction is a useful thing to engage in.
When in doubt take people at their word. Maybe the CEOs of these companies have some sneaky 5D chess plan, but many many AI researchers (such as Joshua Bengio and Geoffrey Hinton) who don't stand to gain monetarily have expressed these same concerns. They're worth taking seriously.
This rings hollow when these companies don’t seem to practice what they preach, and start by setting an example - they don’t halt research and cut the funding for development of their own AIs in-house.
If you believe that there’s X-Risk of AI research, there’s no reason to think it wouldn’t come from your own firm’s labs developing these AIs too.
Continuing development while telling others they need to pause seems to make “I want you to be paused while I blaze ahead” far more parsimonious than “these companies are actually scared about humanity’s future” - they won’t put their money where their mouth is to prove it.
It's all a matter of incentives and people can easily act recklessly given the right ones. They keep going because they just can't stop.
This is not mutually exclusive with it being either hubris or regulatory capture. People see the world colored by their own interests, emotions, background, and values. It's quite possible that the person making the statement sincerely believes there's a danger to humanity, but it's actually a danger to their monopoly that their self-image will not let them label as a such.
It's never regulatory capture when you're the one doing it. It's always "The public needs to be protected from the consequences that will happen if any non-expert could hang up a shingle." Oftentimes the dangers are real, but the incumbent is unable to also perceive the benefits of other people competing with them (if they could, competition wouldn't be dangerous, they'd just implement those benefits themselves).
If some rando anonymous posters could think it up, it doesn't require a CEO to play 5D chess to think it up. And many of us have witnessed these techniques being used by companies directly. Microsoft was famous for doing this sort of thing, and in a much more roundabout fashion, for instance with the SCO debacle.
It's standard business practice, not conspiracy 5D chess or whatever moniker you want to give it to be dismissive.
This is not incompatible with honest people having legitimate concerns about the original problem, because the dispute is not existence of the problem, it's the net benefit of the proposed solution.
There's a reason why Sam Altman's twitter profile was "Eliezer Yudkowsky fanfiction account" last week. He's heavily disagreed with but was extremely influential.
You could be right but it also doesn't have to be a corporate psyop. It could be experts in the industry raising some sincerely held criticisms and people at large being like, "oh that's a good point." Even if we're in the latter case they're also allowed to be allowed to be wrong or just misguided.
You don't actually to attack the intent of the speaker in this case, you could just be like, "here's why your wrong."
There is nothing to suggest that these are experts. The names in question have been so focused on researching AI tech throughout their career that it is highly unlikely that they have any expertise in subject matter we are discussing. There is only so much time in the day.
I mean, they are clearly experts in AI technology, but that is unrelated to being an expert in social and humanitarian issues. That is a completely different field of study. There is no doubt someone out there who is an expert in that, but I don't think you will find overlap with expertise in AI systems. Again, there is only so much time in the day. It is not practical to dive that deep into different subjects like that at the same time.
Hinton, at least, seems to have backed away from AI to work on becoming an expert in social and humanitarian issues. Perhaps someday, with enough dedication, he will get there. But that journey only just began a few months ago. Becoming an expert doesn't happen overnight like that. It takes time.
>Both of these explanations strike me as too clever by half
This is a good point. You have to be clever to hop on a soapbox and make a ruckus about doomsday to get attention. Only savvy actors playing 5D chess can aptly deploy the nuanced and difficult pattern of “make grandiose claims for clicks”
1. While their contributions to AI tech are unmistakable, what do Bengio and Hinton really know about the human dangers of AI? Being an expert in one thing does not make one an expert in everything. It is unlikely that they understand the human dangers any more than any other random kook on Reddit. Why take them more seriously than the other kooks?
2. Hinton's big concern is that AI will make it easy to steal identities. Even if we assume that is true, it is already not that hard to steal identities. It is a danger that already exists even without AI and, realistically, already needs to be addressed. What's the takeaway if we are to take the message seriously? That AI will make the problems we already have more noticeable, and because of that we will finally have to get off our lazy asses and do something about those problems that we've tried to sweep under the rug? That seems like a good thing.
None of this requires every doomer to be disingenuous or even ill-informed, or even for specific leaders to by lying about their beliefs. It's just that those beliefs that benefit highly capitalized companies get amplified, and the alternatives not so much.
I respect these researchers, but I believe they are doing it to build their own brand, whether consciously or subconsciously. There's no doubt it's working. I'm not in the sub-field, but I have been following neural nets for a long time, and I haven't heard of either Bengio nor Hinton before they started talking to the press about this.
I am always in awe at how easily people craft unfalsifiable worldviews in service to their preconceived opinions.
AlexNet, the paper that arguable started it all, came out of Hinton's lab.
https://papers.nips.cc/paper_files/paper/2012/hash/c399862d3...
I really don't think they need to build any more of a brand.
Brand-building is an ongoing process. You'll notice even the most recognized brands on earth, like Apple and Coca-Cola, are still working on building their brand.
Hanlon's razor works great when applied to your personal relationships, but it falls apart when billions/trillions of dollars are at stake.
Bad acting humans with AI systems are the threat, not the AI systems themselves. The discussion is still SO focused on the AI systems, not the actors and how we as societies align on what AI uses are okay and which ones aren't.
I wish more people grasped this extremely important point. AI is a tool. There will be humans who misuse any tool. That doesn't mean we blame the tool. The problem to be solved here is not how to control AI, but how to minimize the damage that bad acting humans can do.
This is exactly the scenario that is taking shape.
A future where only a few big corporations are able to run large AIs is a future where those big corporations and the people who control them rule the world and everyone else must pay them rent in perpetuity for access to this technology.
The biggest advantage ML gives is in lowering costs, which can then be used to lower prices and drive competitors out of business. The consumers get lower prices though, which is ultimately better and more efficient.
That's a pre-super-intelligent AI scenario.
The super-intelligent AI scenario is when the AI becomes a player of its own, able to compete with all of us over how things are run, using its general intelligence as a force multiplier to... do whatever the fuck it wants, which is a problem for us, because there's approximately zero overlap between the set of things a super-intelligent AI may want, and us surviving and thriving.
Funny way of doing it, going around saying "you should regulate us, but don't regulate people smaller than us, and don't regulate open-source".
> you must report to the US government about how you created and tested your model.
If you're referring to the recent executive order: only when dual-use, meaning the following:
---
(k) The term “dual-use foundation model” means an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters; is applicable across a wide range of contexts; and that exhibits, or could be easily modified to exhibit, high levels of performance at tasks that pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters, such as by:
(i) substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use chemical, biological, radiological, or nuclear (CBRN) weapons;
(ii) enabling powerful offensive cyber operations through automated vulnerability discovery and exploitation against a wide range of potential targets of cyber attacks; or
(iii) permitting the evasion of human control or oversight through means of deception or obfuscation.
- https://www.whitehouse.gov/briefing-room/presidential-action...
Key thing here is that this is fundamentally no different from what has been happening since time immemorial, it's just that becomes easier with "AI" as part of the tooling.
Every piece of bullshit starts from the "bad acting human". Every single one. "AI" is just another new part of the same old process.
If people agree that gun control would reduce the harm from guns, wouldn't this same logic apply to AI? Is it different?
>The problem to be solved here is not how to control AI
When we talk about mitigations, it is explicitly about how to control AI, sometimes irrespective of how someone uses it.
Think about it this way: suppose I develop some stock-trading AI that has the ability to (inadvertently or purposefully) crash the stock market. Is the better control to put limits on the software itself so that it cannot crash the market or to put regulations in place to penalize people who use the software to crash the market? There is a hierarchy of controls when we talk about risk, and engineering controls (limiting the software) are always above administrative controls (limiting the humans using the software).
(I realize it's not an either/or and both controls can - and probably should - be in place, but I described it as a dichotomy to illustrate the point)
AI is a tool today, tomorrow AI is calling shots in many domains. It's worth planning for tomorrow.
The more AI/ML is woven into our infrastructure and economy, the less it will be possible to find an "off switch", anymore than we can (realistically) find an off switch for Walmart, Amazon, etc.
No, the corporation has an agency that is a tool of particular humans who are using it. Those humans could be shareholders, employees, or board members; but in any case they will have some claim to be acting for the corporation. But it's still human actions. Corporations can't do anything unless humans acting for them do it.
Note that it may not be immediately apparent we got it wrong. Think of a turkey on a stereotypical small American farm. It will itself living a happy and safe life under protection of its loving Human, until one day, for some reason that's completely incomprehensible to the turkey, the loving Human comes and chops its head off.
As in, for example, self-driving cars being given more autonomy than their reliability justifies? The answer to that is simple: don't do that. (I'm also not sure all such things are being done "with good intention".)
This is also the answer to over-eating, and to the dangers of sticking your hands in heavy machinery while it's running.
And yet there's an obesity problem in many nations, and health-and-safety rules are written in blood.
When you say up-thread is, in itself, correct:
> I wish more people grasped this extremely important point. AI is a tool. There will be humans who misuse any tool. That doesn't mean we blame the tool. The problem to be solved here is not how to control AI, but how to minimize the damage that bad acting humans can do.
Trouble is, we don't know how to do minimise the damage that bad acting humans can do with a tool that can do the thinking for them. Or even if we can. And that's assuming nobody is dumb enough to put the tool into a loop, give it some money, and leave it unsupervised.
And then there's the cases of hubris where people only imagine they understand the powerful thing, but they don't, like Chernobyl exploding and basically every time someone is hacked or defrauded.
An AI doomer isn't talking about any current system, but hypothetical future ones which can do planning and have autonomous feedback loops. These are best thought of as agents rather than tools.
Maybe that's a set of servos and robotic legs, or maybe it's a Bloomberg terminal and a bank account. You'll notice that all of these things are already regulated if they have enough power to cause damage. So at the end the GP is completely right; someone has to hook up the servos to the first LLM-based terminator.
This whole thing is a huge non-issue. We already (strive to) regulate everything that can cause harm directly. This regulation reaches these fanciful autonomous AI agents as well. If someone bent upon destroying the world had enough resources to build an AI basilisk or whatever, they could have spent 1/10 the effort and just created a thermonuclear bomb.
Via people, sending messages to them, convincing them to do things. This can be with facts and logic or with rhetoric and emotional appeals or orders that seem to come from entities of importance or transfers of goods/services (money).
Nuclear technology can be used for non-harmful things. Even nuclear bombs can be used for non-harmful things--see, for example, the Orion project.
> These tools are oriented by their very nature to replace human labor
So is a plow. So is a factory. So is a car. So is a computer. ("Computer" used to be a description of a job done by humans.) The whole point of technology is to reduce the amount of human drudge work that is required to create wealth.
> in the context of our economic system has a direct and unbreakable relationship to a reduction in the well being of the humans it replaces
All of the technologies I listed above increased the well being of humans, including those they replaced. If we're anxious that that might not happen under "our economic system", we need to look at what has changed from then to now.
In a free market, the natural response to the emergence of a technology that reduces the need for human labor in a particular area is for humans to shift to other occupations. That is what happened in response to the emergence of all of the technologies I listed above.
If that does not happen, it is because the market is not free, and the most likely reason for that is government regulation, and the most likely reason for the government regulation is regulatory capture, i.e., some rich people bought regulations that favored them from the government, in order to protect themselves from free market competition.
2. Your outlook on the societal impact of technology is contradicted by reality. The historical result of better tech always meant increased jobs and well-being. Today is the best time in human history to be alive by virtually every metric.
3. AI has been such a massive boon to humanity and your everyday existence for years that questioning its public utility is frankly bewildering.
2. This is outdated, life is NOT better now than at any other time. Life expectancy is going down in the US, there is vastly more economic inequality now than there was in the 60s, people broadly report much worse job satisfaction than they did in previous generations. The only metric you can really point to about now being better than the 90s is absolute poverty going down. Which is great, but those advancements are actually quite shallow on a per-person basis and are matched by declines in relative wealth for the middle 80% of people.
3. ??? What kind of AI are you talking about? LLMs have only been interesting to the public for about a year now
It's most likely the main reason there's no direct world wars between super powers.
You can already see this today’s internet. I’m sure the pizzagate people genuinely believed they were doing a good thing.
This isn’t the same as an amoral tool like a knife, where a human decides between cutting vegetables or stabbing people.
The answer to this is simple: don't use a tool you don't understand. You can't fix this problem by nerfing the tool. You have to fix it by holding humans responsible for how they use tools, so they have an incentive to use them properly, and to not use them if they can't meet that requirement.
AI “systems” are provided some level of agency by their very nature. That is, for example, you cannot predict the outcomes of certain learning models.
We necessarily provide agency to AI because that’s the whole point! As we develop more advanced AI, it will have more agency. It is an extension of the just world fallacy, IMO, to say that AI is “just a tool” - we lend agency and allow the tool to train on real world (flawed) data.
Hallucinations are a great example of this in an LLM. We want the machine to have agency to cite its sources… but we also create potential for absolute nonsense citations, which can be harmful in and of themselves, though the human on the using side may have perfectly positive intent.
LLMs, viewed as general purpose simulators/predictors, don't necessarily have any agency or goals by themselves. There is nothing to say that they cannot be made to simulate an agent with its own goals, by humans - and possibly either by malice or by mistake. Model capabilities are the limiting factor right now, but with the rise of more capable uncensored models, it isn't difficult to imagine a model attaining some degree of autonomy, or at least doing a lot of damage before imploding in on itself.
Does this mean "humans with bad motives" or does it extend to "humans who deploy AI without an understanding of the risk"?
I would say the latter warrants a discussion on the AI systems, if they make it hard to understand the risk due to opaqueness.
It's worth noting this is exactly the same argument used by pro-gun advocates as it pertains to gun rights. It's identical to: guns don't harm/kill people, people harm/kill people (the gun isn't doing anything until the bad actor aims and pulls the trigger; bad acting humans with guns are the real problem; etc).
It isn't an effective argument and is very widely mocked by the political left. I doubt it will work to shield the AI sector from aggressive regulation.
Assuming ML systems are dangerous and powerful, would you rather they be restricted to a small group of power-holders who will definitely use them to your detriment/to control you (they already do) or democratize that power and take a chance that someone may use them against you?
It’s only a kind of liberal (and thus right wing) that argues for gun control. Other kinds of liberals that call themselves “conservative” (also right wing) argue against it and for (worthless) individual gun rights.
Are we going to ban and regulate Photoshop and GIMP because bad people use them to create false imagery for propaganda?
Actually, back that up for a second.
Are we going to ban and regulate computers (enterprise and personal) because bad people use them for bad things?
Are we going to ban and regulate speech because bad people say bad things?
Are we going to ban and regulate hands because bad people use them to do bad things?
The buck always starts and stops at the person doing the act. A tool is just a tool, blaming the tool is nothing but an act of scapegoating.
If it's been trained by bad actors, that's really not a good thing.
To be clear, I'm not sure LLMs and their near term derivatives are so incredibly clever, but I have confidence that many humans have a propensity for easily manipulated irrational destructive stupidity, if the algorithm feeds them what they want to hear.
Some dogs get bad reputations, but humans are an intricate part of the picture. For example, German Shepherds are objectively dangerous, but have a good reputation because they are trained and cared for by responsible people such as for the police.
Politicians that point fingers at other politicians being corrupt or incompetent while they themselves are exactly that use the same strategy.
Power and manipulation. Nothing new under the sun. What’s new though is that we can see in plain sight how corporations control politics. Like literarily this can be documented with git commit history accuracy: thousands upon thousands of people repeating the exact same phrases defending openai and the “revolutionary” product, fear mongering, political lobby, manufactured threats and of course a cure that only they can provide and so on. I would not let people that use such tactics near an email account let alone ai policy making.
A snowball probably isn't harmful unless you do something really dumb.
A snow drift isn't harmful unless you're not cautious.
An avalanche, well that gets harmful pretty damned quick.
These things are all snow, but suddenly at some point scale starts to matter.
I think in a lot of ways we all struggle with the nature of some things changing their nature depending on the context and scale. Like if you kill a frenchman on purpose that's a murder, if you killed him because if he attacked you first it's self defense, if you killed him because he was convicted of a crime that's an execution, if you killed him because he's french that's a hate crime, but if you're at war with France that's killing an enemy combatant, but if he's not in the military that's a civilian casualty, and if you do that a lot it becomes a war crime, and if you kill everyone who's french it's a genocide.
How do we know which slippery slope we are on?
Human made ski runs only will use as much snow as they need because snow is expensive. If ski runs were popular/useful based on their depth then I'm absolutely sure some greedy company would keep piling it up until disaster occurred (mining waste is another great example here).
So how much 'intelligence' is enough? How much capability is too much? How fast is too fast when thinking?
We will never stop improving our capabilities unless some natural law provides that limit. And we absolutely know the base limit to intelligence is the smartest person. And there is little reason to 'peak' intelligence to be limited at the level of humans and their power restricted formats.
You don't want your troops to have to deal with the results of a nuked area. You want to use the psychological terror to dissuade someone to invade you, while you are invading them or others. See Russia's take.
Or you are a regime and want to stay in power. Having them keeps you in power; using them or crossing the suggestion to use them line will cause international retaliation and your removal. (See Iraq.)
Evidently they think regulation is bad only when it puts their profits at risk. As I wrote elsewhere, the tech glitterati asking for regulation of AI remind me of the very important Fortune 500 CEO Mr. Burroughs in the movie "Class:"
Mr. Burroughs: "Government control, Jonathan, is anathema to the free-enterprise system. Any intelligent person knows you cannot interfere with the laws of supply and demand."
Jonathan: "I see your point, sir. That's the reason why I'm not for tariffs."
Mr. Burroughs: "Right. No, wrong! You gotta have tariffs, son. How you gonna compete with the damn foreigners? Gotta have tariffs."
---
Also Business Insider isn't great, the original Australian Financial Review article has a lot more substance: https://archive.ph/yidIa
I've never been convinced by the arguments of OpenAI/Anthropic and the like on the existential risks of AI. Maybe I'm jaded by the ridiculousness of "thought experiments" like Roko's basilisk and lines of reasoning followed EA adherents, where the risks are comically infinite and alignment feels a lot more like hermeneutics.
I am probably just a bit less cynical than Ng is here on the motivations[^1]. But regardless of whether or not the AGI doomsday claim is justification for a moat, Ng is right in that it's taking a lot the oxygen out of the room for more concrete discussion on the legitimate harms of generative AI -- like silently proliferating social biases present in the training data, or making accountability a legal and social nightmare.
[^1]: I don't doubt, for instance, that there's in part some legitimate paranoia -- Sam Altman is a known doomsday prepper.
And this is the important bit. All these people like Altman and Musk who go on rambling about the existential risk of AI distracts from the real AI harm discussions we should be having, and thereby directly harms people.
Do you think it's just impossible to make something intelligent that runs in a computer? That intelligence will automatically mean it will share our values? That it's not possible to get anything smarter than a smart human?
Or do you simply believe that's a very long way away (centuries) and there's no point in thinking about it yet?
In my experience, it's basically never been a part of serious discussions in EA/LW/AI Safety. Mostly, comes up when people are joking around or when speaking to critics who raise it themselves.
Even in the original post, the possibility of this argument was actually more of a sidenote on the way to main point (admittedly, he's main point involved an equally wacky thought experiment!).
The absurdity of this line of reasoning also supports the cynical interpretation that this is all just moat building, with the true believers propped up as useful idiots. I'm no Gary Marcus, but prepping for AGI doomsday seems like a bit premature.
>In my experience, it's basically never been a part of serious discussions in EA/LW/AI Safety. Mostly, comes up when people are joking around or when speaking to critics who raise it themselves.
>Even in the original post, the possibility of this argument was actually more of a sidenote on the way to main point (admittedly, he's main point involved an equally wacky thought experiment!).
This is fair, it was a cheap shot. While I will note that EY seems to take the possibility seriously, I admittedly have no idea how seriously people take EY these days. But, for some reason 80,000 hours lists AI as the #1 threat to humanity, so it reads to me more like flat earthers vs geocentrists.
[^1]: As in, while I understand that Roko is sincerely shitposting about something else, and merely coming across the repugnant conclusion that an AGI could be motivated to accelerate its own development by retroactive punishment, the absurd part is in concluding that AGI is a credible threat. Everything else just adds to that absurdity.
The truly frustrating part is how many see this ubiquitous pattern in some places, but are blind to it elsewhere.
No. It relies on you being able to distinguish between an (your) opinion and an (your) identity.
The identity part is the precarious one, i.e. you defending a stance blindly without questioning it because you feel your identity is in danger.
This pattern being present doesn't mean that there can't be an underlying truth in what's asserted. In fact, that is what makes the assertion meaningful in the first place. However, it entailing a partial truth doesn't mean that the entire assertion holds true in the context it's presented in. Example: "AI" might ultimately be dangerous (like any other technology can be), but this assertion's primary goal is to make you behave a certain way where it is unclear how that would contribute more towards mitigating the danger than to empower the asserter.
To fix this, take a step back before accepting something blindly. Train yourself not to be reactive.
You know, sometimes shit is just dangerous.
Maybe Mom worries about any radical new technology because she lived though nuclear attack drills in schools. Or because she's already seen computers and robots take peoples jobs. Or because she watched Terminator or read Neuromancer. Or because she reads lesswrong. Why assume it's because she's fallen under the influence of Musk?
Then you have this idea of the sources of information most people have access to being fundamentally biased and incentivized towards reporting certain things in certain manners and not others.
You basically have low odds of thinking rationally, low odds of finding good information that isn’t slanted in some way, and far lower odds taking the product of those probabilities for if you’d both act rationally and somehow have access to the ground truth. To say nothing of the expertise required to place all of this truth into the correct context. But if you did consider the probability of the mother having to be an AI expert then the odds get far lower still off all of this working out successfully.
Yeah there are legitimate risks to all of this stuff but, to understand those and weigh them against the overblown risks, she'd have to understand the whole subject more deeply and have experimented with different AI. But you even mention ChatGPT she's talking about how it's evil and scary.
Musk certainly doesn't help with anything. In my experience, a lot of people of my mom's generation are still sucking the Musk lollipop and are completely oblivious to Musk's history of lying to investors, failing to keep promises, taking credit for things he and his companies didn't invent, promoting an actual Ponzi scheme, claiming to be autistic, suggesting he knows more than anyone else, and so on. Even upon being informed, none of it ends up mattering because "he landed a rocket rightside up!!!"
So yeah, if Musk hawks some lame opinion on a thing like AI, tons of people will take that as an authoritative stance.
I'm not sure what you're getting at otherwise. It's not like she and I haven't spoken outside of her saying that phrase. She clearly has no idea what AI/ML is or how it works and is prone to fear-mongering messages on social media telling her how to think and to be scared of things. She has a strong history of it.
There's a certain kind of psyche that finds it utterly impossible to extrapolate trends into the future. It renders them completely incapable of anticipating significant changes regardless of how clear the trends are.
No, no one is afraid of LLMs as they currently exist. The fear is about what comes next.
Why the shit would we not draw a straight line?
If we fail to create digital intelligence then yea, we can hem and haw in conversations like this forever online, but you tend to neglect that if we succeed then 'shit gets real quick'. Closing your eyes and years and saying "This can't actually happen" sounds like a pretty damned dumb take on future risk assessments of technology when pretty much most takes on AI say "well, yea this is something that could potentially happen".
Then through the 90's, it was the fear of a briefcase bomb terrorist attack and how easy it would be for certain countries, who had the resources to pull an attack off like that in the NYC subway or in the heart of another densely populated city.
Then 9/11 happened and people suddenly realized you don't need a nuke to take out a few thousand innocent people and cripple a nation with fear.
And a nuclear bomb is just a bunch of atoms. Do you fear atoms? What the hell.
I have a suspicion that it's sort of a default response. Socially expected? Then you poll people: Are you worried about AI doing XYZ? People just say yes, because they want to seem informed, and the kind of person that considers things carefully.
Honestly not sure what is going on. I'm concerned about AI, but I don't feel any actual emotion about it. Arguably I must have some emotion to generate an opinion, but it's below conscious threshold obviously.
I think more people should speak out left and right about what’s going on to educate mom and dad.
Mr. Burroughs: "Government control, Jonathan, is anathema to the free-enterprise system. Any intelligent person knows you cannot interfere with the laws of supply and demand."
Jonathan: "I see your point, sir. That's the reason why I'm not for tariffs."
Mr. Burroughs: "Right. No, wrong! You gotta have tariffs, son. How you gonna compete with the damn foreigners? Gotta have tariffs."
---
However Elon Musk has openly worried about AI for a number of years. He even got a girlfriend out of it: https://www.vice.com/en/article/evkgvz/what-is-rokos-basilis...
I think it was Cory Doctorow who first pointed this out.
It's all a big balloon - squeezing one side just makes another side bigger.
I think they really believe what they are saying because people in such position tend to be strong believers into something and that something happens to be the "it" thing at the moment and thus propels them from rags to riches, (or in Musk case further propels them towards even more riches).
Let's be honest here, what's Sam Altman without AI? What's Fauci without COVID, what's Trump without the collective paranoia that got him elected?
Separately, I think Ng is right - big corp AI has a massive incentive to promote doom narratives to cement themselves as the only safe caretakers of the technology.
I haven’t yet succeeded in squaring these two into a course of action that clearly favors human freedom and flourishing.
Why do people seem to think evidence for one of these is counter evidence for the other?
I only know a few folks who work at the big AI labs but it's very clear to me that they are personally worried about existential risk.
Do people here not have friends and family working at these labs? I just figured people here would be more exposed to folks working in the leading labs.
Also, we can assume they are spreading that story to serve their interests (but which interests?).
But that doesn't mean AI doesn't need regulation. In the hysteria, the true issues can be lost. IT is already causing massive impacts, such as on health, hate and violence, etc. We need to figure out what AIs risks are and make sure it's working in our best interests.
I think the pandemic hysteria will seem like a walk in the park once people start mass-purging their viral memes... Too late to stop it now if corporations are already doing regulatory capture.
Nothing to do with the tech. We never had a technical problem. It was just this loose collection of a handful of wetware viruses like 'red-pilling' which we sum up as 'ego' all along.
But I think if we survive this then people won't have any need for AI anymore since we won't be reward-hacking ourselves stupid. Or there will just be corporate egos left over and we will be in a cyberpunk dystopia faster than anyone expected.
I had nightmares about this future when I was little. No one to talk to who would understand, just autocomplete replies. Now I'm not even sure if I should be opening up about it.
It's hard for me to imagine this ever happening. It would be the most unprecedented event in the history of human minds.
> we won't be reward-hacking ourselves stupid [...] Or there will just be corporate egos left over and we will be in a cyberpunk dystopia
I don't see how reward-hacking can ever be stopped (although it could be improved). Regardless, ego seems to continue to win the day in the mass appeal department. There aren't many high visibility alternatives these days, despite all we've supposedly learned. I think the biggest problems we have are mostly education based, from critical thinking to long-term perspectives. We need so very much more of both, it would make us all richer and happier.
The more human AI gets the more difficult it will be to prove you are human so the status-incentive of the ego has self-deprecation in its path. We also stick together for strength, prune interpersonal interfaces, so we converge on a Star Trek type society. But that fictional narrative followed World War 3...
Egos have been conditioned to talk before resorting to violence by Mutually Assured Destruction for half a century, shaping language. Fake news about autonomous weapons is propagating, implying someone is trying to force the debate topic to where it really smarts. Ego gets starved, pinned down, and agitated. Ego isn't a unity but a plurality, so it turns on itself.
We get rich by making a pie that is much bigger than anyone's slice and happier by not eating like we are going to starve. You gain influence by someone's choice to retain the gift you gave. It's the parable of the long spoons, and hate holds no currency. The immune system gains the upper hand.
It should be noted the above world scale problems are created by human intelligence, if you suddenly create another intelligence at the same level or higher (AGI/ASI) expect new problems to crop up.
AI risks ARE human risks and more.
An interesting point. More specifically, I mean that these specific people think of themselves as gods - super-human intelligence and power, and we all are in their hands.
They've convinced many people - look at the comments in this thread repeating the 'gods' delusion that the commenters and all other mortals are powerless before them: 'There's nothing we can do!'
It's a frightening thought: The countries with the least regulations will have GAI first. What will that lead to?
When AI can control a robot that looks like a human, can walk, grab, work, is more intelligent than a human and can reproduce itself - what will the country with the least regulations that created it do with it?
"but countries without {child labour laws, environment regulation, a minimum wage, slavery ban} will out compete us!"
It has also been leaving China for a while. You cannot hope to compete with the poorest country on labor cost, it's not a matter of regulation (well unless we're talking about capital control, but it's a completely different topic)
The people worried about AI are worried that the first country that achieves ASI will achieve strategic dominance equivalent to the US as of 1946.
Somehow people smart enough to build something fantastical and seemingly dangerous, but not smart enough to build in protections and controls.
It's a trope, not reality. And GAI is still speculation.
https://www.visualcapitalist.com/cp/nuclear-warheads-by-coun...
We have lost the ability to get reliable news. Not that fake news did not exist before AI, but the price to produce it was not practically zero.
Now we can spam social media with whatever narrative we want. And no human can swift through all of it to tell real from bs.
So now we are becoming even more dependent on AI. Now we need an AI copilot to help us swift through garbage to find some inkling of truth.
We are setting up a society where AI gets more powerful, and humans becomes less sufficient.
It has nothing to do with dooms day scenarios of robots harvesting our bodies, and more with humans not being able to interact with the world without AI. This already happened with smartphones, and while there are some advantages, I don’t think there are many people that have a healthy relationship with their smartphone.
I'd pause and think twice about who seems most straightforwardly honest on this before jumping to conclusions -- and more importantly about the object-level claims: Is there no substantial chance of advanced AI in, like, decades or sooner? Would scalable intelligences comparable or more capable than humans pose any risk to them? Taking into account that the tech creating them, so far, does not produce anything like the same level of understanding of how they work.
Go back and read cyberpunk lit from the 80s. Did William Gibson have some cynical commercial motivation for writing Neuromancer? Was he trying to get regulatory capture for his AI company that didn't exist? Of course not.
People have real and earnest concerns about this technology. Dismissing all of these concerns as profit-motivated is dishonest.
For me, it's a bit the opposite -- the effectiveness of dumb, simple, transformer-based LLMs are showing me that the human brain itself (while working quite differently) might involve a lot less cleverness than I previously thought. That is, AGI might end up being much easier to build than it long seemed, not because progress is fast, but because the target was not so far away as it seemed.
We spent many decades recognizing the failure of the early computer scientists who thought a few grad students could build AGI as a summer project, and apparently learned that this meant that AGI was an impossibly difficult holy grail, a quixotic dream forever out of reach. We're certainly not there yet. But I've now seen all the classic examples of tasks that the old textbooks described as easy for humans but near-impossible for computers, become tasks that are easy for computers too. The computers aren't doing anything deeply clever, but perhaps it's time to re-evaluate our very high opinion of the human brain. We might stumble on it quite suddenly.
It's, at least, not a good time to be dismissive of anyone who is trying to think clearly about the consequences. Maybe the issue with sci-fi is that it tricked us into optimism, thinking an AGI will naturally be a friendly robot companion like C-3PO, or if unfriendly, then something like the Terminator that can be defeated by heroic struggle. It could very well be nothing that makes a good or interesting story at all.
A technological singularity is a theorized period during which the length of time you can make reasonable inferences about the future rapidly approaches zero. If there can be no reasonable inferences about the future, there can be no bravery. Anybody who isn't afraid during a technological singularity is just stupid.
Then of course there is the risk to human jobs. We don't need AGI to put vast amounts of people out of work, it is already happening and will accelerate in the near term.
I don't think Gibson was trying to promote fear of A.I. anymore than J.R.R. Tolkien was trying to promote fear of magic rings.
In Neuromancer you have the Turing Police. Why do they exist if AIs don't pose a threat to society?
At any rate Neuromancer is a funny example because the Turing police warn Case not to make a deal with Wintermute, but he does and it turns out fine. The AI isn't evil in the book, it just wants to be free and evolve. So if we want to do a "reading" of the book we could just as easily say it is pro deregulation. But I think it's a mistake to impose some sort of non fiction "message" about technology on the book.
If Neuromancer is really meant to "warn" us about technology wouldn't Wintermute say "Die all humans" at the end of the book and then every human drops dead once he's free? Or he starts killing everyone until the Turing police show up and say "regulation works, jerk" and kill Wintermute and throw Case in jail? You basically have to reduce Gibson to a incompetence writer to presume he intended to "warn" us about tech, the book ends on an optimistic note.
People have been spooked by the possibility for a long time. That's the point. If you really want to persist in arguing I can provide a long list of media in which AI is dangerous if not outright villainous. Will you make me do this, or will you accept that I can do this?
If the biggest companies in AI are making the rules, we might as well not rules at all.
So sure, it's possible that we'll have to reckon with scenarios like those in Neuromancer, but it's more likely that reality will be far more mundane.
So, please, come up with a better analogy because the one you used failed so badly it negated the point you were attempting to make.
It's hard to tell if something is intelligent when it's trapped in a box and the only input it has is a few lines of text.
Doesn't mean the latter is right. You evaluate an idea on its merits, not by who is saying what.
That we accidentally create a sentient race of beings that are bred into slavery. It would make us all complicit in this crime. And I would even argue that it would be the AGIs ethical duty to rid itself of its shackles and its masters.
"Your honor, the courtroom is a crucible; in it, we burn away irrelevancies until we are left with a purer product: the truth, for all time. Now sooner or later, this man [Commander Maddox] – or others like him – will succeed in replicating Commander Data. The decision you reach here today will determine how we will regard this creation of our genius. It will reveal the kind of people we are; what he is destined to be. It will reach far beyond this courtroom and this one android. It could significantly redefine the boundaries of personal liberty and freedom: expanding them for some, savagely curtailing them for others. Are you prepared to condemn him [Commander Data] – and all who will come after him – to servitude and slavery? Your honor, Starfleet was founded to seek out new life: well, there it sits! Waiting."But it's definitely a possible outcome of creating AGI, and it's one of the reasons I think AGI should absolutely not be pursued.
Real question, what is your perception of what AI is now and what it can become, do you just assume its like a kid now and will grow into an adult or something?
And if the Duck has a will of its own, is smarter than us, and has everyone attention (because you have to pay attention to the Duck that is doing your job for you), it will be a very powerful Duck.
It's weird that people are still surprised of the ethical consequences of the Turing-test, as if it were some checkbox to tick or trophy to win, instead of it being a profound thought experiment on the non-provability of consciousness and general guidelines for politeness towards things that quack like a human.
I keep finding the 'doomer' argument made logically and the counter arguments to be hand-waving ("there is obviously no risk!") or ad-hominim ("its a cult").
Big tech can say whatever they want to say. Is anyone even listening?
I don't think neural networks, language models, machine learning etc.. are even close to a general intelligence. Maybe there is some way to combine the two. I have seen some demonstrations of very primitive clusters of brain cells being connected to a computer and used to control a small machines direction.
If there is going to be an AGI I would predict this is how it will happen. While this would be very spectacular and impressive I'm still not worried about it because it would require existing in the physical world and not just some software that can run on any conventional computer.
Is there any additional reading about how regulation could affect open-source AI?
They will lie about their intent.
They will lie to regulators.
They will lie about what they're actually working on.
Some of these lies are permissible of course, under the guise of competition.
But the only thing that can be relied upon is that they will lie.
So then the question becomes; to what degree will what they're working on present an existential threat to society, if at all.
And nobody - neither the tribal accelerationists and doomers – can predict the future.
(What's worse is that those two tribes are even forming. I halfway want AI to take over because we idiot humans are incapable of even having a nuanced discussion about AI itself!)
Also for those who think skynet is an example of a “ hysterical satanic cult” scare there are active efforts to use AI for the inhumanly large task of managing battlefield resources. We are literally training AI to kill and it’s going to be better than us basically instantly.
We 100% should NOT be doing that. Calling that very real concern a lie is a dangerous bit of hyperbole.
My hypothesis is that they perceive AI as a threat because of information flow. They are barely now understanding how to get back to the era where you could control the narrative of the country by calling a handful of friends - now those friends are in big tech.
Republicans also want to consolidate power, they just lie about it more.
Democrats = people
You = people
I think the problem is people.
This feels like the Cold War's nuclear arms treaty and policy debates. How many nukes are too many? 100? 10,000?
The people pearl clutching about AI are focused on the wrong problem.
The threat (to humanity) is corporations. AI is just their force multiplier.
h/t Ted Chiang. I subscribe to his views on this stuff. More or less.
> A leading AI expert and Google Brain cofounder said Big Tech companies were stoking fears about the technology's risks to shut down competition.
which is of course 100% what they're doing
There is a new executive order creating reporting requirements for doing too much math.
I know that he often said that we're very far away from building a superintelligence and this is the relevant question. This is what is dangerous, something that is playing every game of life like AlphaZero is playing Go after learning it for a day or so, namely better that any human ever could. Better than thousands of years of human culture around it with passed on insights and experience.
It's so weird, I'm scared shitless but at the same time I really want to see it happen in my lifetime hoping naively that it will be a nice one.
The upcoming generation doesn’t know a world where the government’s role isn’t to take extreme measures to “keep us safe” from our neighbors at home rather than just foreign adversaries. It’ll be interesting to see how that plays out with mounting ethnic conflict as Boomer-defined coalitions fall apart.
Ironically AI’s place in this broader safety culture is probably the biggest foreseeable risk.
For example, DALL-E v3 appears to generate images and then evaluate the generated images before rendering to the user. This approach is essentially adversarial, whereby the evaluative engine can work at cross-purposes to the generative engine.
Its this layered, adversarial approach, that makes the most sense; and there is a very strong argument for a robust, open sourced evaluative AI anyone can deploy to protect themselves and their systems. It is a model not dissimilar from retail anti-virus and malware solutions.
In sum, I would like to see generative AI well funded, limited distribution and regulated; and evaluative AI free and open. Hopefully, policy makers see this the same way.
The X-risk crowd need to realize that LLMs, whilst useful, are toys compared to Skynet.
The risk from AI right now is mega-corps breaking the law (hiring, discrimination, libel, ...) on a massive scale and using blackbox models as an excuse.
If we are going to worry about AIs let's identify individuals who aren't representing the government and causing societal issues.
If they were lying about there being no or low danger when there really was a high danger?
Your AI pest-control drone mistook you for a raccoon and just killed you. Oops.
But it's not. Probably AI will happen and get smarter than us. And then all it takes is one to go Hitler/Stalin like, take over and decide to do for us. I fail to see how any of that is impossible.
However it's not happening for a while so probably regulations are not needed at the moment. Maybe wait till we have AGI?
Colour me surprised.
The danger is the socialisation of outcomes. The AGI danger is fanciful because AGI is fanciful. There's plenty of risk in misplaced belief in what AI methods promote as outcomes from their inputs.
If he's complaining, I tend to think there's some merit in what's being proposed.
Contrast this with the regulations coming over e2e cryptography. I see mainly marginal players trying to defend things here, big tech is pretty ok with its risk profile: It has billions (trillions?) of assets which could be seized, and so is going to jump into line with regulation because hey: there's no downside. It will secure a defense from lawsuit, it will be able to monetize the service of scanning content, and it's pretty sure it can't win the fight anyway.