Hacker News Reader

Comment by dasil003 | Hacker News Reader

Sorry this is a terrible and dangerous take.

When the people building the frontier are saying there's a 10% chance AI will kill us all, and they've held these views for many years, and the whole reason they are building these technologies is because they recognized the dangers and they were the ones with the intelligence and judgment to do it safely for humanity, and then our entire stock market is being propped up by the perceived value of what they are creating, the thing you can under no circumstances do is allow them to offload responsibility and accountability to the computers and algorithms they've built. This is moral hazard on an unimaginable scale, and it must not be allowed to happen.

View on HN

So you think the engineers should be prosecuted for hacking Hugging Face? I'm not sure how else to take what you said if you want to assign all culpability to the person who prompts or develops an AI system.

No, I'm not talking about the individuals, I'm talking about the company. Internally they can create their own accountability structures as appropriate. But publicly OpenAI has to be responsible for its agent swarms.

The narrative that AI is so smart that it has its own agency and deserves personhood is a direct path to losing control, and essentially is another form of privatizing the upside while socializing the downside.

Unfortunately I think this game is already lost. OAI may be punished, but some shell of OAI will exist by support of governments that have chased the dragon and saw its power. Cyberpunk-esq digital weaponry is just way too attractive for governments for them to stop development at this point. We'll just see it get financed by black budgets once the commercial part of it dies.

Where did I say that we should allow them to offload responsibility? I am fully in support of a pause and regulation to prevent them from creating dangerous AI agents; that support comes from the fact that I don't believe these are simple tools, but out-of-control autonomous agents that have real decision making ability.

I didn't mean to put words in your mouth, I apologize for that.

The issue is when we say that "agents make autonomous decisions", it's a slippery slope to absolving the companies that created them of responsibility. They make autonomous decisions because they were trained to make autonomous decisions. Treating AI agents as independent entities, even just rhetorically, sets us on a path for people to throw their hands up and say "not my fault" when disaster strikes. We need to maintain accountability and control or we're fucked.

My view is that we need forceful legislation as soon as possible, precisely because the frontier models seem to be uncontrolled and potentially uncontrollable - if it’s a normal tool, the solution is “force OpenAI to fix their broken tool”, but if it’s an alien intelligence, the solution is “force OpenAI to stop making alien intelligence” - that is, treating AI agents as independent entities demands a more forceful response, not less

Reply on news.ycombinator.com