Hacker News Reader

Comment by SpicyLemonZest | Hacker News Reader

> It's a strange way for any company to talk about its own work. You don't hear McDonald's announcing that it's created a burger so terrifyingly delicious that it would be unethical to grill it for the public.

> Here's one theory.

But the author never gets back to this! It's the main observation the theory has to account for; why don't we see other companies speak this way, if it's such an effective strategy for deflecting non-apocalyptic concerns?

View on HN

I think they get away with it because it's a dual-use technology. They have this tool that could end the world and people want in on it because they want the power.

The answer to the burger analogy is that it's the wrong analogy. McDonald's is selling you the burger. AI companies are essentially selling you the grill.

The hype works so well because it plays on people's ego and desire for power. They think I have the power to end the world with this technology but I won't because I'm a good person.

> why don't we see other companies speak this way

They do. Every company who promised us that their shitty cell phone app or website was going to change the world and revolutionize and disrupt industry/society was guilty of the same thing. They just usually focused their ridiculous levels of hype on the positives. The goal was the same. "Our technology is going to change the world so investors had better give us cash or else they will be left behind" is still the message.

I think this is just an advancement of what we saw with self-driving cars and how companies were pushing narratives around how every trucker will be out of work (this still hasn't happened) or how no individuals would own a car again while deflecting from things like how badly their cars performed in snow/rain or in anything other than very carefully controlled and mapped out conditions.

There's no past tense "saw", self-driving cars are still a thing! Waymo announced that they're expanding to Portland yesterday (https://waymo.com/blog/shorts/waymo-in-portland/), and the announcement does not include anything but sunshine and roses. Even within the AI space, I really don't see anyone other than frontier AI labs talking about their product this way.

If McDonald's food was featured in sci fi movies about being able to end humanity through war, that's when this would apply and they'd cultivate fear of that nonsense to distract from their food being shitty and overpriced and unhealthy.

tbf most companies don't have a potentially world-ending product. only real similar field is defense contractors who typically can't brag about unreleased ideas as they're classified.

I agree, but the experts the author cites do not. Professor Valor believes that AI is a mirror and any existential fears of it are just reflected fear of ourselves; Professor Bender believes that AI is a con and all the people who say it's powerful enough to be world-ending are lying. Anyone who concedes the premise that AI has a genuine potential to be world-ending is, I think, on the AI labs' side of this debate.

We are not such a bad thing to fear.

This technology interacts socially, so even if it can't jailbreak itself on a technology-level (which feels like a tough guarantee to make at this point) it can simply ask someone to do a bad thing and there is some chance they'll do it. The same way a human leader does.

The first kids who have only faint memories of a time before chatbots will be entering the military in 6-7 years. You have to assume they are acting as best friends, therapists, or even surrogate parents for a substantial number of kids right now.

We are going to need years to figure out what to do about this technology. I think some impetus to get that process started is a good idea.

Reply on news.ycombinator.com