Hacker News Reader

Comment by stephbook | Hacker News Reader

We've had movies like Terminator, Matrix, or i, Robot. Black Mirror Metalhead. Boston dynamics robo dog with a knife or a gun. Humanoids doing back flips. Memes making fun of how we are going to fight these for access to water.

The intelligence is there and so is the physical incarnation.

Google's models fold proteins better than humans and Covid might have been engineered and inadvertently escaped from a lab.

But when CEOs warn of that, they're dismissed as scare mongers. Why? Are only powerless internet commentators allowed to be fearful?

View on HN

They aren’t attempting to self-regulate. On the contrary, it seems like they’re doing whatever they can to bring about any possible threat as fast as possible in order to get legislation. Regulatory capture is what everybody is saying is their goal and I think that’s easier to believe

They are obviously trying to self regulate. That's the whole point of the calls to pace the frontier. See Anthropic's launch post for Opus 5.5 where they got third party sign off on its safety, or see the launch post for Astra where OpenAI explicitly calls out the safety improvements made on HuggingFace-esque evals. Self regulation won't help if other labs don't play ball, though.

> Self regulation won't help

You got it

They are obviously trying to self-regulate and correctly realize that competitive pressures will render these efforts ineffective, and have communicated this to everyone.

Kind of makes you wonder why they built the tech in the first place, I don't know if we're on the eve of the apocalypse or not, but why the fuck build it all and then run around telling us how dangerous it is when...they were already warned?

Bit strange no?

I'm a bit fatalistic about it all now. The cat is out of the bag, unless we see some never seen before leadership from the current US admin there is almost zero point in worrying about it. Not going to pretend what the future holds but it's looking unpredictable, as usual :)

What do you mean? The benefits are obvious. Curing most death and disease, and solving most technical problems in one fell swoop. Just because people talk about the risks of a thing doesn't mean they don't think there are benefits!

> The benefits are obvious. Curing most death and disease, and solving most technical problems in one fell swoop.

I dont see how these two arw obvious. If nothing else, people running things dont care about actual harm done to real people. Their idea of safety is "making sure humans merge with ai and we will have transhumanist society in far away future".

With those priorities, short term stuff like "less people dying this year" dont matter.

> If nothing else, people running things dont care about actual harm done to real people

Seems like unwarranted cynicism to me. If they didn't, they wouldn't give a shit about discussing the risks of AI. You should actually talk to some people at the labs.

> ...competitive pressures will render these efforts ineffective...

Competitive pressures from whom, exactly? Based on his public statements, Altman seems to be confident that he can instruct OpenAI to halt R&D while they try to figure out just how exactly their computer-hacking tool managed to access the Internet and hack computers this time. [0]

Plus, the only US-based LLM manufacturers that are doing bleeding-edge work are all simultaneously screaming that they need to be stopped before they commit even more crimes, and/or maybe eliminate all of humanity. On top of that -if memory serves- Anthropic's CEO called to exclude China from the conversation about regulating the development of LLMs, so he clearly believes that China need not be part of the coordination on this problem. Maybe that's because the official line from these companies is that the LLMs coming out of China are 100% ripoffs of Anthropic's and OpenAI's work... so -clearly- cutting off China's access will solve that problem?

[0] Pro tip: It's because the network that the machines running the software is connected to permits traffic from those machines out to the Internet. See [1] for some more discussion on the topic.

[1] <https://news.ycombinator.com/item?id=49862136>

But when CEOs warn of that, they're dismissed as scare mongers. Why? Are only powerless internet commentators allowed to be fearful?

Well are they going to stop developing the tech or not? If not, then you can see why people don't trust what they're saying. I mean if it's that dangerous, why not step down from your role as CEO to demonstrate how seriously you take the threat? Why not immediately call a pause on all training, today ?

If there is one gun in the room, who do you trust most to hold it?

Only suckers say "not me".

> But when CEOs warn of that, they're dismissed as scare mongers. Why? Are only powerless internet commentators allowed to be fearful?

Because they conveniently skip the part that investors don't like: that liberal free market capitalism will be destroyed by AI.

holy heck, thank you for the moderating dose of sanity. HN comments feel like a death cult lately. All I can think is that this is intelligence addiction speaking.

I say this as an intense user of these tools. But they should rightfully be understood as terrifying, and their continued frontier development is imho not safe.

It’s crazy to read all the comments basically dismissing everything. It’s either complete regulatory capture or no regulation at all. How about at least some reasonable regulation? Fine, assume the CEOs don’t care about safety and want regulatory capture.

We can ignore that and push for something reasonable. In other industries such as housing construction we have building codes without making housing construction a monopoly. We don’t just let random people build nuclear power plants and then go “if you think it’s dangerous, stop doing it.” People are going to do dangerous things even when they know it’s dangerous. They also know it’s dangerous.

I don't see people arguing against some regulation. I see people arguing against basing your regulation on the narrative sold by openai and anthropic.

Reply on news.ycombinator.com