Comment by ipdashc | Hacker News Reader

Comment by ipdashc | Hacker News Reader

Parent Full thread

> When a bridge collapses and people die, then we say the same: "someone should be in jail". The different with the software industry is that nobody gives a shit about the harm they cause. Civil engineers have to, or they go to jail.

The difference is that 99.9% of software is just not as important as even your average two-lane bridge.

The recent cries to send random OpenAI people to jail frankly strike me as IT people just inflating our own importance. Oh no, HuggingFace got hacked. So what? Nobody got hurt, no individual lost their property. A company had to rebuild some server configs. The reason nobody cares enough to send someone to jail is that it's just not that important.

Also, even civil engineers very rarely go to jail when a building or bridge collapses. This is something everyone is using as an analogy when in reality it's very hard to prove criminal negligence.

> The point here being that maybe, just maybe, AI labs should not be shipping stuff they cannot secure.

Yeah, and when they say "okay, fine, we should slow down" everyone says they're making it up for marketing reasons.

And then if they actually slow down on their own everyone makes fun of them for being obsolete and switches to their competitor.

Tricky problem, isn't it?

> The difference is that 99.9% of software is just not as important as even your average two-lane bridge.

You miss the point. LLMs are not the two-lane bridge here, they are the trigger. Sure, when an LLM makes 99.9% of software fail, nobody cares. But what happens when it touches a single, very critical instance of software?

> Oh no, HuggingFace got hacked. So what?

Ever heard of NotPetya? Blocked Maersk unwillingly (as a collateral). Could have resulted in a global economical crisis. There is a nice Darknet Diaries episode about that.

> Also, even civil engineers very rarely go to jail when a building or bridge collapses.

At the very least they don't go public saying "well it happens, building bridges is hard". Usually it goes more "it shouldn't have happened, and we need to get our act together and make sure it doesn't happen again anytime soon".

> Tricky problem, isn't it?

I get the irony. And I also totally get why someone getting an insane salary would see no reason to not work on it (and cynically think "if I am not getting rich having fun here, someone else will... may as well be me"). Who am I to say I wouldn't do the same? I strongly believe that sending astronauts to space is fun but useless, and taxpayers shouldn't agree. But offer me to go to space? I go tomorrow. Ironic, isn't it?

I guess my point is that when you get rich having fun working something you love, maybe you don't go whining online telling people that you missed your daughter's birthday and that they should not be pissed at the potential damage you may be contributing to doing. And comparing yourself to Sully (while saying you don't).

> But what happens when it touches a single, very critical instance of software?

I don't disagree at all, but the AI companies/engineers to their credit have been very clear on that risk - if anything, a bit too "clear" with all the apocalyptic rhetoric. And I've been disappointed that the overall response has either been "they should go to jail" (for a set of incidents that, while useful canaries in the coal mine, effectively harmed nobody) or "they're making it all up for marketing/monopolization". Sure, it's different people saying both, but it still feels kind of dumb that this is somehow collectively our response as a community.

Admittedly I'm pretty much on your side here, I just think the "yeahh send them to jail!" thing in particular is peak "internet ruining discussion on an important topic by taking it to its most ridiculous conclusion"

> I guess my point is that when you get rich having fun working something you love, maybe you don't go whining online telling people that you missed your daughter's birthday and that they should not be pissed at the potential damage you may be contributing to doing.

Agreed, it's pretty hard to try and get sympathy out of that. But on the other hand, what else can they do? I'd rather they explain their side of things than just stay quiet about it. They are right on their main point, which is that it's a hard problem at its core and I can understand why getting hit with a barrage of "just configure a sandbox dude" would tick anyone off.

Yeah I think we agree. I just wanted to note this:

> but the AI companies/engineers to their credit have been very clear on that risk

I am not a big fan of that. It's too easy to say "I make a ton of money harming society, but I'm good because I am telling me that you should regulate me". If someone told me "I hit my wife, but you should not criticise me because it is not illegal, and I have been telling for years that it should be illegal. But as long as it is legal I will hit her", I wouldn't suddenly respect them just because they are apparently very well aware that what they do it bad.

Fine with those engineers explaining how it is on their side, but not whining because other people think what they do is unethical. It sounds like "please, I need you to tell me that I am a good person even though I take money (and pride) contributing to something I know may hurt society".

Reply on news.ycombinator.com