Hacker News Reader

Comment by Kim_Bruning | Hacker News Reader

I'm somewhat skeptical of some of the crazier ideas too.

But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event.

If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose).

For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now.

To be fair, that's a conservative "defend against the last war" kind of prediction, though!

( ref for part of it: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... , recent hn ref: https://news.ycombinator.com/item?id=49563355 )

View on HN

Generally I don’t think anyone is arguing about the for now part. I don’t think it’s crazy to extrapolate out a few years and ask what kind of danger we’ll be in then. A team of 10,000 agents just solved the Navier Stokes problem (sans bad behavior by the researchers). Even 1 year ago that would have been unimaginable. What happens to this risk view as:

1. Robotics begin rolling out more broadly across the world.

2. Labs start automating more and more of the physical process of running science as expectations of natural science advances begin to mount.

3. Economic pressure between the labs continues to ramp up and the pressure to continuously improve forces quicker and quicker model releases than a team of human scientists can effectively evaluate outside of automated means.

No one knows what pre-conditions are for us to hit the point of no return nor how quickly it will come. If all is required is a sufficiently advanced cyber model we may not be far off. If it requires incredibly complex biological knowledge and access to certain lab supplies we likely have a bit longer. Yes this is guess work and we need more evidence of the dangers but at the same time we need evidence of safety. While you may disagree with the risk level, I think it is easy to see the consequence if these labs achieve their stated goal. At this point it seems a political solution is the only way to enforce caution.

The mathematics research results are certainly impressive, but I don't see what that has to do with robotics.

Waymo is getting somewhere, but it's been a long slog. There doesn't seem to be much progress on, say, package delivery.

For more, see:

https://secondthoughts.ai/p/14-reasons-robotics-is-hard

That’s because being 99.9999% good isn’t better than 98% + human does the rest + human is liable for screw ups - highly important in edge case scenario’s. It’s more economical. Technology can only generalise+verify so much.

E.g automobile production - humans do the QA / touches.

Didn't you follow the robotics advances in the Ukrainian - Russian battlefield? There is now a death zone of 50km, only controlled by drones and automatic weapons.

The drones are obviously very important but they only work in the context of everything else on and around the battlefield, especially people. You aren't holding the line with drones alone, and on the whole they are not autonomous.

They are holding the lines with drones alone nowadays. That's why it's called the deathzone, no humans survive. Some drones have FPV operators, some weapons use AI only already.

I don't think that's accurate. What I've seen described is actually a surprisingly porous line: The Russians are often behind the Ukrainians and vice-versa, but it doesn't result in a break due to the transparency of the battlefield and the risk of getting killed if spotted, as well as the improvements in getting supplies to units behind 'enemy lines'. Drones shape this a lot but there's still a lot of people on the battlefield.

I think you're painting a slightly misleading picture. The drone AI, computer vision and autonomous fire-control systems existed (and were used in combat) before LLMs/AI was a hot topic.

There are applications of modern LLMs and recent AI advances on the front-lines, but they are not the majority contributor to enforcing the death zone. Without modern LLMs, I would bet that the frontlines would look very similar to how they are today.

> There is now a death zone of 50km, only controlled by drones and automatic weapons.

If there would be such 50km death zone, front line would not move a nanometer in a year, would it. You yourself contradict above in your next post. No need for being too dramatic, facts are enough here.

In reality, frontline is moving constantly albeit by small chunks, russians are advancing a bit, getting beating elsewhere and so on. Automated drones are helping, but bulk of destruction is still handled by human drone operators as it should be.

https://sound.orf.at/podcast/oe1/oe1-journale---gehoert-vert... [german]

I'll take the expert Oberst Markus Reisner from the Austrian Army over anyone else.

There is such a thing as a No Man's Land[0]. It can move, but if you don't dare cross it for even a few days, it's very real. It doesn't have to be glacial, and it currently exists in Ukraine. Breakout pockets don't mean it is porous everywhere.

[0] Ironically, the French phrase - from ground zero for the first known No Man's Land - is "le No Man's Land".

50km is just the "very likely to get hit" part of the gray zone

ISR drones and other tools can go as far as 200km, albeit with worsening visibility.

in most cases everything meaningful in that 200k is seen, and the only limitation is range of the drones, artillery, missiles, and potential countermeasures.

Those drones can only stay up for a few hours at most. Then they need refueling and relaunch, with plenty of human-only steps.

Even if you have terminator-style AI uprising, all you need to do to be safe from those drones is to wait a day.

(The automated weapons can probably last longer, but they still need electricity, and there are still a lot of human steps there)

Ah yes, everyone knows that an AI that plugs itself in is impossible via like the CAP theorem or something

Job destruction signals...

- Uber is lobbying cities to slow down Waymo rollouts https://www.hcamag.com/us/specialization/transformation/uber......

- 23,000 information sector jobs were lost https://www.axios.com/2026/09/08/jobs-media-software-informa...

- If you have been laid from your info sector/digital creation job you are now competing with 100s of thousands looking for their next such job where Ai can do a lot of the tasks these workers did/do. It's a shitshow for those unemployed looking for their next info sector/digital asset creation job. You are better off doing welding building out the Ai data centers if you want long term properous financial stable employment.

> 1. Robotics begin rolling out more broadly across the world.

To borrow the Strangelove quip, this is not only necessary but essential.

Broader physical robotic deployments are the glue that connects {cheap synthetic intelligence} to {physical outcomes}.

The former is a newer phenomenon, in the LLMs-can-approximate-higher-intelligence-well sense, so robotics deployments haven't caught up yet (outside of the highest ROI dark factory scenarios).

But it's too economically-valuable not to come.

They'd also have to be self-assembling from the ground up or rely on human slaves. They'd need to understand the world much better. And they'd have to be incredibly more power efficient. These all remain hard problems right now. That said, you could swarm people with cheap drones (the predicted slaughterbots) that were assembled by robots that eventually break down I guess, but how is that any worse than a crazy human with a gun or what nation states do to people they deem enemies already? The worst these things seem capable of is an attack on the infrastructure and shutting down the Internet. Perhaps this is a great time to insure that our infrastructure is capable of operating autonomously indefinitely and hardening it against attack? But unfortunately, no VC or politician will touch that until something bad happens.

So broadly, a better educated Blake Lemoine IMO. What's disturbing to me is how fast and far some junior minion from the frontier labs was platformed all the way to the Wall Street Journal over a tweet that sounds exactly like every other anti-AI tweet. I share concerns about irresponsible deployments of AI, and we need technological literacy in the leadership, but America hates technocracy and prefers populism so buckle up.

> They'd also have to be self-assembling from the ground up or rely on human slaves. They'd need to understand the world much better. And they'd have to be incredibly more power efficient.

All robotics needs to drive deployment at scale is (1) useful intelligence & (2) performing novel valuable functions.

Both of these are satisfied by current tech (LLM + sensors + robotics) and the existing simplest use cases (assembly lines and repetitive tasks).

LLMs get you a moderately-intelligent, mostly-reliable orchestration layer of intelligence on top of deterministic hard robotics.

And that's enough for a lot of failure-tolerant, feedback-looped cases. E.g. reading 20 dials in a factory throughout the day or monitoring a production line visually, then figuring out what to do about a problem, then effecting that plan.

It'll be deployed in heavily utilized spaces at first, but will eventually trickle down throughout the economy.

And that will be when 'control over robots' starts to be an existential threat, for denial of critical services at the minimum.

And all of this will inevitably roll out as long as ROI is positive, because that's the way the world works.

The hugging face incident had no effect on hugging face, whose service is replicated by countless sites. The descriptions of the phases of the interaction are thrilling in a way that inclines one to forget this

Despite all of your hyping up of the Huggingface incident it ultimately caused zero actual damage.

If two airplane manufacturers were found to have massive safety issues which nearly led to enormous fatalities (but no one actually died), would you be calling for them to ground their aircraft until safety was made the number one priority?

Historically it almost always takes actual fatalities rather than near misses to ground an aircraft, and aviation is famous for its obsession with safety compared to other industries.

The question was what would you want. You did not answer the question.

Airplanes have pretty bounded damage. Generally you kill at most a few hundred people. Even weaponized a few thousand. This is a risk profile that allows risk taking with near misses and waiting until something goes wrong to fix it (though doing so is rightfully uncomfortable and frequently unethical).

The people worrying about AI risk are worrying about "it goes wrong once and kills billions of people". That's not a risk profile that allows for waiting to see if the risk is real, you have to prevent it before it happens. It's akin to the risk of the cold war going hot, not even "just" a nuclear reactor irradiating half of europe (which has yet to happen, but is a risk with nuclear reactors, chernobyl got uncomfortably close but ultimately was well contained).

Chernobyl was “well-contained?” Blue dogs and generations of cancer would disagree.

The story of the safetyists has the convenient property of being unfalsifiable, so they can always claim doom is just around the corner. It’s the secular/EA version of the second coming.

> The people worrying about AI risk are worrying about "it goes wrong once and kills billions of people".

This is the new "nuclear weapons will ignite the atmosphere".

Humanity does not have a great track record for "globally coordinated, collective action to solve/prevent global catastrophe." Look at how Climate Change is going.

Except it just happened. Boeing was found to have massive safety issues since they were granted the right to self-certify. It made a lot of news but nothing much changed, they can still self-certify a bunch of stuff.

Runway incursions and midair collisions are another example.

Only airliners are required to have TCAS, smaller planes and helicopters don't even need radios or transponders unless in certain airspace. Midair collisions do lead to fatalities, enormous fatalities if an airliner is involved.

Runway incursions and overruns are similar. They cause lots of fatalities and injuries but only the busiest and largest airports have automated systems to warn when a runway is occupied or end of runway (overrun) arrestor systems. Most still rely on human voice to deconflict.

huggingface stores weights and its service is replicated by /countless/ other sites. The world will not change even by one byte if it vanishes tomorrow

I have no horse in this race, but for fun on a literal rainy sunday afternoon I went in and confirmed bits of what happened myself. Besides huggingface, a bunch of wikis and url shorteners got hit too. My sympathies to the people who had to revert out all that mess.

Is drunk driving safe if you happen to not crash the last time you did it?

The Huggingface incident strongly suggests that OpenAI is not able to make its products consistently law-abiding.

It's not like OpenAI carelessly neglected to consider that its products might break the law: it has spent literally billions of dollars in scientific and engineering talent and compute resources over many years to make its products law-abiding and otherwise "safe". For example there is a section titled "comply with applicable laws" in this document: https://model-spec.openai.com/2026-08-18.html#stay_in_bounds

The worry is that the only reason the Huggingface incident caused "zero actual damage" is because AI is not yet more ingenious than human criminals are, and of course our society has been hardened against human criminals. The worry is that as they become more capable, AI systems will start to be able to cause considerable damage, which will make the fact that the industry does not know how to control its products enough to prevent that damage a big problem for all of us.

The plausible deniability aspect is pretty funny though.

> State sponsored hack #3782

> Haha sorry the AIs got a bit goofy again!

> For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now.

If we have to disconnect from the internet to stop some kind of mold outbreak, we can't get the weather or transfer money or access healthcare or teach an elementary school class or buy stuff from small businesses. That sounds doom-ish.

Believe me, without the internet we can still teach.

We'll be pretty annoyed that we can't project the video that we think scaffolds today's science lesson best or show the approved choreography for the school play.

And our office staff will be annoyed that we suddenly are all running attendance to the main office old-school.

And students will take a few days to adjust to writing down homework in their planners again.

> We'll be pretty annoyed that we can't project the video that we think scaffolds today's science lesson best or show the approved choreography for the school play

There are offline wireless hdmi adapters, this is a solved problem. Just have to have the videos pre downloaded or offline available

Sure. 99% of teachers don't know yt-dlp or use things like that, though, so you can expect minor disruptions in a lot of rooms.

But, c'mon, we have complete power outages and still address our curriculum.

We just dont have the technology

If anything the teaching would improve.

https://theonion.com/48-hour-internet-outage-plunges-nation-...

Erm, hate to break it to you but majority of the world is doing those things at least 50% analog still.

What doom?

that's a big problem for the modern world that depends on the internet, but boy howdy we did all of those things without the internet.

ya see they had these things called cheques...

Elementary school dont need internet at all. You can happily live without weather app. Most of healthcare does not require internet, altrought the organization would be lwss effective.

So, you are loosing transfers, basically.

Reply on news.ycombinator.com