Prediction: AI Will Collapse | Hacker News Reader

Prediction: AI Will Collapse

twitter.com

19 pointsasdev14 comments

The actual post doesn't really match the title, unless we're talking about AI stocks, which it didn't go into much.

The real point is that AI agents, for people rather than for coders, must be 100% bulletproof. As the post said, people have to be able to use them while drunk. They have to be able to use them while distracted.

Coding agents? The users are, presumably, both sober and paying attention. The general public? They won't be.

> The real point is that AI agents, for people rather than for coders, must be 100% bulletproof

Seems wrong. Nothing is 100% bulletproof. We'll just adapt our social structures to deal with agent mistakes the same way we deal with human mistakes.

Exactly this. The point of human management is not to do subordinates' jobs, rather to ensure that subordinates are performing their jobs to some standard of adequacy.

Yes, I don’t know where this frame came from. Humans adopt useful systems. Providers attempt to limit exposure to liability. Society will navigate the opposing pressures and fallout in differing ways according to local jurisprudence, customs, and norms. AI is just too damn useful (and fun for now).

> Coding agents? The users are, presumably, both sober and paying attention.

This user is tired of clicking through to allow trivial commands to run, while being less and less engaged with the task at hand, but left with no alternative given the volume of changes others ram through.

That's the thing. I was once that guy who was meticulous about reviewing code and later making sure to approve every little change an agent wanted to make. But I mostly gave up because of the business demand to use AI to ram as many features through as fast as possible. I don't go full blown YOLO, but I do run agents in highly controlled VM instances and keep them fairly autonomous. Being asked questions or for permission every few minutes like the carpenter from that Seinfeld episode is a recipe for insanity.

You can take a more involved approach to AI assisted development. Work at the line/function completion level, rather than vibing inside a harness. Zed offers a lot of this style of development out of the box, and you can use Copilot inside VSCode for a similar experience, or use a regular harness inside a terminal pane in VSCode but discipline yourself to small, targeted edits. As a human you have the strongest right to be the loop. Be the orchestrator.

> The users are, presumably, both sober

heh... the Ballmer Peak is still going strong, even with AI.

Now I wonder if my agent(s) can detect when I'm coding drunk.

> In the same way that YouTube videos have a 0% chance of phishing, cross-site scripting, malware, and viruses.

So, the same as in not actually 0%? Even though the video streams themselves were never used for attacks[0], the contents of the videos frequently have been. The closest analogue to phishing would be the fake "SpaceX launch" streams that were actually a carrier for a crypto scam payload. Or for malware, it doesn't matter that the video itself wasn't malware as long as a video showing a fake Fortnite hack could get the viewer to install malware thinking they got the hack.

AI agents need to be substantially better than humans at this, but they don't need to be perfect.

[0] As far as I know! It seems very hard to smuggle a vuln past the transcoding, but I guess technically possible.

> It must be impossible for AI to: make purchases you did not intend, send messages you did not intend, and access information you did not want to share.

Hrm. I'd say that all of these things are possible when using services of companies that are valued at billions/trillions in the stock market. So "impossible" might be too strict.

Yes, being able to dispute charges and having a good return policy makes up for a lot of things.

People were wary about entering their credit card number into a website at first, but they got used to it, and new payment systems were built.

> First: it thought that html and javascript were the means by which common people would express themselves. The dot com bubble thought everyone would have a website. As it turns out, normal people prefer expressing themselves with text, images, and video.

So, I just took a look at blogger.com, instagram.com, and youtube.com. They all seem to run on html and javascript. If someone thought normies were going to write javascript by hand that was never, ever going to happen and is not a tick against html/js. In fact, the dotcom bust is what made html and js more robust. I have lots of money creating html/js for people and businesses.

> Html is not a good form of self-expression. It’s hard to secure. It’s hard to distribute. It’s hard to reason about. You need to learn how to code.

HTML is no worse than Word, paper, or the printing press. It's not hard to secure at all because html carries no risks, it's a markup language. It's not hard to distribute at all, not even a tiny bit. I don't even know what one would "reason about" with html. Do they also struggle to "reason about" paper? What exactly IS "reasonsing about" html? Learn to code? Are you for real? In 2026? People haven't been forced to write html by hand for almost 30 years.

> Second: it missed that 90% of the internet’s traffic would come through video.

No, no one missed this. There's an easy way to prove it, too. Look at all the video sites created 20+ years ago. Hulu, Netflix, Youtube, etc. 20+ years ago. No one missed video being the next big thing except this author.

> Not correct: > Fiber usage would scale with number of websites > People in the network would scale with number of websites

I agree these are not correct, but that's like saying "the moon is a rabbit that likes to hum Mozart" is wrong. Yes, it's wrong, but no one ever argued that was true.

> It would be more true to say: > Fiber usage would scale with video streaming > People in the network would scale with social media adoption

It would, and these are the things everyone DID predict.

This whole post is nonsense.

i think first comment x summrize this well. you are thinking too small.

Reply on news.ycombinator.com