Hacker News Reader

Comment by wduquette | Hacker News Reader

I anticipate a lengthy series of catastrophes initiated by AI agents with too much power and too little sense. Today the agent accidentally deletes your repo; tomorrow we lose a day's worth of banking transactions; Friday the cell phone network goes down. Such things, if sufficiently destructive, could result in widespread hunger, etc.

But killing all humans? Not likely.

View on HN

> But killing all humans? Not likely.

I hope you're right. But the US dept of war is using grok. [1]

I don't really trust them to use it responsibly. That is to say, I expect those using it to just go along with whatever it says.

And it's grok... not even one of the good models

[1] https://www.war.gov/News/Releases/Release/Article/4586482/de...

Maybe, but AI will enable actions that were never before possible and that are an existential threat. Consider microbiology: automated viral/bacterial research. microbes could be created before humans have a chance to even consider the potential harm. Pandemics, food chain collapse, ecosystem collapse are all possible.

Isn't nature doing exactly that every single day?

You are not seriously suggesting that nature and AI-accelerated gain-of-function research are similar in regards to risks, are you?

This isn't nature: https://news.ycombinator.com/item?id=49628290

Nature's "made" natural nuclear reactors (Oklo), but we had to make the hydrogen bomb ourselves.

https://blog.genesmindsmachines.com/p/im-sorry-youre-not-goi...

https://www.defenseone.com/technology/2026/09/ai-powered-bio...

It occurs to me that letting AI agent process your incoming e-mail really isn't that different in principle than letting incoming e-mail include scripts to run at your console. Why are people so eager to rush into this brave new world?

Reply on news.ycombinator.com