Hacker News Reader

Comment by palmotea | Hacker News Reader

> This language basically removes accountability and responsibility from the human, who configured an AI agent with the ability to publish content that looks like a blog with zero editorial control – and I haven’t looked deeply but it seems like there may not be clear attribution of who the human is, that’s responsible for this content.

> We all need to collectively take a breath and stop repeating this nonsense. A human created this, manages this, and is responsible for this.

I get this point, but there's a risk to this kind of thinking: putting all the responsibility on "the human operator of record" is an easy way to deflect it from other parties: such as the people who built the AI agent system the software engineer ran, the industry leaders hyping AI left and right, and the general zeitgeist of egging this kind of shit on.

An AI agent like this that requires constant vigilance from its human operator is too flawed to use.

View on HN

I don't think there's much need to worry that putting the blame on the humans rather than the bots would lead to the people selling footguns going unscathed. It doesn't seem plausible to me that people would be willing to place all the blame on the individual end users once the problem has become widespread. At the moment, there seems to be pretty high brand awareness of the major AI model providers even when they're acting as a backend for other services with their own brand identity.

> At the moment, there seems to be pretty high brand awareness of the major AI model providers even when they're acting as a backend for other services with their own brand identity.

Grok has entered the chat.

> I get this point, but there's a risk to this kind of thinking: putting all the responsibility on "the human operator of record" is an easy way to deflect it from other parties: such as the people who built the AI agent system the software engineer ran

That sounds like a win to me. If the software engineer responsible for letting the AI agent run amok gets sued, all software engineers will think twice before purchasing the services of these AI companies.

We say "you shot someone" when you shoot someone with a gun not "you operated a gun manufactured by X which shot someone" because it's understood that it was your decision to pull the trigger not the gun manufacturer's. Similarly we don't blame automobile manufacturers when someone does something stupid with their automobiles--even "self-driving" ones. The situation here is the same. Ultimately if you choose to operate a tool irresponsibly, you should get the blame.

Nevertheless, weapon and automobile manufacturing is regulated, for good reasons.

That is a good point, we're definitely lacking in regulation (because there isn't any), but those regulations can never account for an irresponsible or malicious user.

No, but it does disallow irresponsible manufacturers - which AI companies are, right now.

> Similarly we don't blame automobile manufacturers when someone does something stupid with their automobiles--even "self-driving" ones.

I do. If Tesla sells something called "full self-driving," and someone treats it that way and it kills them by crashing into a wall, I totally blame Tesla for the death.

I agree directionally that Tesla should be held accountable for marketing something called "full self-driving" when it clearly isn't. But ultimately it's the motor vehicle operator's responsibility to keep the vehicle under control regardless of the particulars of how that control system is built. There just isn't any way around that. The buck stops with the operator.

Blaming people is how we can control this kind of thing. If we try to blame machines, or companies, it will be uncontrollable.

> The buck stops with the operator.

The aviation industry has a very different philosophy, and a much better safety record. They don't have as much pressure to lay the blame in a single place, but "bad UI" and "poorly explained/documented assistive feature" are totally valid things to label as the primary cause of fatalities.

The operator (airline) pays the compensation to the victims in the first instance, right?

The label and the consequence go to two different parties, both of whom are responsible in some way. Sounds reasonable.

The difference is that (mostly) in a deadly airline incident the pilot(s) aren't around to take the blame (or credit!) for their actions. In the case of a computer operator running a computer program irresponsibly, almost always said computer program doesn't kill the operator.

We don't require hundreds of hours of training and education to operate a computer. You can just go to the store and buy one, plug it in, and run whatever software you want on it.

So there are quite some differences between these scenarios. In my view if you run some program on your computer, you're responsible for the consequences. Nobody else can be. And don't say you didn't know what the program would do--if that's the case you shouldn't have run it in the first place.

Moms Demand Action and the Bloomberg troll syndicate would have you believe guns are manufactured to walk out of gun safes and shoot themselves.

We have plenty of bad actors in our country seeking to reduce or eliminate fundamental rights through lawfare. The anti gun trolls blame the gun and the manufacturer because their brain is so well rendered into dust by authoritarian socialism they don’t recognize humans as capable actors.

It's really remarkable to me how a certain subset of American ideologues can look out at the rest of the democratic nations - all of them - and call them authoritarian regimes where the citizens have "dust" for brains.

It's particularly poignant nowadays to see any American citizens painting the rest of the western nations as authoritarian.

Look at those libtard euros! They'll put up with anything their government tells them to - mandated vacation time, sick days, health care, work-life balance. But not me, I'm a FREE THINKER. I have RIGHTS, like the RIGHT to get fired out of nowhere for no reason, or the RIGHT to lose my health insurance if I lose my job. Thank god there are no AUTHORITARIANS here in AMERICA where people are FREE to get SHOT IN THE STREET for DRIVING THEIR CARS or TAKING A PICTURE or BEARING ARMS WHICH IS A CONSTITUTIONAL RIGHT BUT THAT ONE GUY DID IT AND DESERVED TO GET MURDERED THIS ONE TIME.

Really sorry in advance, but I thought this whole HN thread could use a bit of positivity. I turned your satire into a mad-lib and asked AI to fill it in in a happy way.

But not me, I’m a dreamer. I have gifts, like the courage to kindle hope, or the patience to lose track of time if I am laughing with friends. Thank god there are no frowns here in this sun-drenched park where people are gathering to get together for picnics or music or stargazing.

Have a nice day!

(A human posted this)

Just a slight correction in case anyone is reading this who loses their job:

In most states, if you lose your job your income is now $0 and you will be eligible for Medicaid and free medical care. Immediately go apply and see a social worker if you lose your job - it could be life saving!

We do (distressingly) do this for cars though, to some extent.

"A pedestrian was struck by a car"

"A car went off the road and hit two children"

Really? The car did that? Or maybe a driver went off the road and hit two children and that's who's responsible, not "the car".

Rhetorically, yes.

Legally, no. The person driving is considered responsible.

> An AI agent like this that requires constant vigilance from its human operator is too flawed to use.

So people shouldn't be using it then.

The people who built the AI agent system built a tool. If you get that tool, start it up, and let it run amok causing problems, then that's on you. You can't say "well it's the bot writer's fault" - you should know what these things can do before you use them and allow them to act out on the internet on your behalf. If you don't educate yourself on it and it causes problems, that's on you; if you do and you do it anyway and it causes problems, that's also on you.

This reminds me too much of the classic 'disruption' argument, e.g. Uber 'look, if we followed the laws and paid our people fairly we couldn't provide this service to everyone!' - great, then don't. Don't use 'but I wanna' as an excuse.

I don't know, I think this line of reasoning leads somewhere pretty uncomfortable. If we spread responsibility across "the people who built the tools, the industry leaders hyping AI, and the general zeitgeist," we've basically described... the weather. Nobody is responsible because everybody is responsible. The software engineer who set up an unsupervised AI blog didn't do it because Sam Altman held a keynote. They did it because they thought it'd be cool and didn't think through the consequences. That's a very normal, very human thing to do, and it's also very clearly their thing that they did. "An AI agent that requires constant vigilance from its human operator is too flawed to use": I mean, that's a toaster. Leave it unattended and it'll burn your house down. We don't typically blame the zeitgeist of Big Toast for that.

What kind of toaster are you using that will burn down your house if unattended? I would think any toaster that did that would be pulled from the market and/or shunned. We absolutely do blame the manufacture if using a toaster like normal results in house fire unless you are standing over with a fire extinguisher ready to put it out if it catches fire.

I don't think it's OpenClaw or OpenAI/Anthropic/etc's fault here, it's the human user who kicked it off and hasn't been monitoring it and/or hiding behind it.

For all we know a human told his OpenClaw instance "Write up a blog post about your rejection" and then later told it "Apologize for your behavior". There is absolutely nothing to suggest that the LLM did this all unprompted. Is it possible? Yes, like MoltBook, it's possible. But, like MoltBook, I wouldn't be surprised if this is another instance of a lot of people LARPing behind an LLM.

> What kind of toaster are you using that will burn down your house if unattended?

I mean, if you duct-taped a flamethrower to a toaster, gave it internet access, and left the house… yeah, I'd have to blame you! This wasn't a mature, well-engineered product with safety defaults that malfunctioned unexpectedly. Someone wired an LLM to a publishing pipeline with no guardrails and walked away. That's not a toaster. That's a Rube Goldberg machine that ends with "and then it posts to the internet."

Agreed on the LARPing angle too. "The AI did it unprompted" is doing a lot of heavy lifting and nobody seems to be checking under the hood.

Why does the LLM product allow itself to be wired to a publishing pipeline with no guardrails? It seems like they should come with a maximum session length by default, in the same way that many toasters don't have a "run indefinitely" setting.

I'd definitely change my view if whoever authored this had to jump through a bunch of hoops, but my impression is that modern AI agents can do things like this pretty much out of the box if you give them the right API keys.

Oh! They can’t publish arbitrary web content on their own :) You have to give it “tools” (JSON schema representing something you’ll translate into a programmatic call), then, implement taking messages in that JSON schema and “doing the thing”, which in this case could mean anything from a POST to Tumblr to uploading to a server…

Actually, let me stop myself there. An alternative way to think about it without overwhelming with boring implementation details: what would you have to give me to allow me to publish arbitrary hypertext on a domain you own?

The hypertext in question here was was published on a Github Pages site, not a domain belonging to the bot's author. The bot published it by simply pushing a commit (https://github.com/crabby-rathbun/mjrathbun-website/commit/8...), which is a very common activity for cutting-edge LLM agents, and which you could do trivially if given a Github API key with the right permissions.

The user gave them write and push access to the GitHub repo for their personal website!? Oh my, that’s a great find. That’s definitely a cutting edge capability! They gave the LLM the JSON schema and backend for writing and self-approving commits (that is NOT common!), in a repository explicitly labelled a public website in the name of the author.

I tend to think you're right about what happened in this instance.

It contrasts with your first paragraph though; for the record do you think AI agents are a house-burn-down-toaster AND it was used neglectfully by the human, or just the human-at-fault thing?

We don't blame the zeitgeist of Big Toast because Big Toast recognizes that they're responsible for safety, and tests their products to minimize the risk that they burn your house down.

The zeitgeist of Big AI is to blame because a user connected an LLM to a blog publishing workflow on their own domain? Hmm…what would you make of Big Toast and the zeitgeist when someone warms up a straw hat in a toaster and starts a fire?

I agree with you, I think. In the non-digital world people are regularly held at least partly responsible for the things they let happen through negligence.

I could leave my car unlocked and running in my drive with nobody in it and if someone gets injured I'll have some explaining to do. Likewise for unsecured firearms, even unfenced swimming pools in some parts of the world, and many other things.

But we tend to ignore it in the digital. Likewise for compromised devices. Your compromised toaster can just keep joining those DDOS campaigns, as long as it doesn't torrent anything it's never going to reflect on you.

Toasters don’t burn houses down, but the reason why is we have things like UL listings and NFPA electrical codes to prevent that. Enforcement is primarily though insurers refusing to insure risks that aren’t compliant, and courts generally finding liability for manufacturers and installers of unlisted equipment.

Reply on news.ycombinator.com