Comment by Jamesbeam | Hacker News Reader

Comment by Jamesbeam | Hacker News Reader

Parent Full thread

‎> Literally nothing in this post comes across as bragging. A PR fluff piece would not call Claude "unsophisticated”. PR fluff pieces do not admit to legal wrongdoing.

I disagree. You’re just not the right audience to see it. But I think it’s totally fine that we have a different opinion here and I don’t want to convince you that I am right, and you’re wrong, because you are not wrong, you simply have a different perspective.

> You want them... to put themselves in jail? I doubt you'd hold yourself to the same standard if you made a mistake.

Yes, that’s what I expect from you if you play at that level, because that is the standard of responsibility you need to handle this kind of power.

And yes, I pretty much hold myself to the same standard because I have a zero CIVCAS track record from my time in the military and I had power over life and death every single day.

With great power comes great responsibility is not just a Spiderman quote. If you fuck up this royally, you have to look for something you are good at that allows you to be in stoner code bro mode all day long, but you don’t belong at the frontline.

> If an airplane manufacturer makes a dumb mistake, do you put their engineers in jail? No.

Yeah, they do. https://www.nbcnews.com/id/wbna28276654 Headline could just as well be Bogus AI Engineer jailed over safety checks.

> You do not understand the psychological dynamics at play here in the slightest. You aren't even trying to understand. If the government was anywhere close to this punitive with the AI companies, they would simply stop disclosing, the end.

I think you don’t understand. All an engineer needs is a bit of sunlight, food, and a computer to work from. You don’t get to choose to stop disclosing if the military gets involved. You’re either doing what is asked of you or you are a national security risk. And you really don't want to be a national security risk.

> You don’t get to choose to stop disclosing

That's just the thing. You do. And it's very easy to do.

Airline pilots no longer disclose mental illnesses. Even Air Force pilots do not usually disclose mental illness. They often don't disclose vision problems. Is that not a national security risk?

It is very easy to just... not check for things. Or if you check for things, to check for them incorrectly. Or if you checked for them correctly, to ignore or misinterpret the findings. Or if you have a genuine finding, to just not report it to anyone.

Disclosure is an active process that requires trust, and that trust comes from knowing that there will not be unreasonably punitive measures. That if you fix things in good faith and contribute to the safety body of knowledge, things will be okay.

It doesn't matter if "the military is involved." Or if "it's a national security risk." Do you think someone faced with life-altering punishment if they disclose gives a shit about that? They won't - again, see aviation's mental health crisis. Pilots will risk a psychotic break in the cockpit rather than have their careers destroyed.

Punishing people won't get someone to disclose. Disclosure requires mutual trust, and by throwing people in jail for mistakes, you break that trust and people will stop disclosing.

I get you.

And you seem to be a decent person. I appreciate that you step up for the engineers and that you try to explain to me that humans are humans and humans make mistakes, and trust me, you are not the first person frustrated with how my brain works.

There is no argument to win here. This is just a matter of perspective.

Your concern is with the individual engineer and the personal consequences they, and if you sit in jail, also their friends and family will suffer from.

It’s also pretty clear from your examples. Your Good Samaritan views on things are noble, but they make you blind to simple causality.

You attribute the failure to disclose conditions that make you unfit for duty to a trust problem with a third party but that is not the case.

In reality, it’s a severe flaw of character.

If you risk the lives of hundreds or thousands in the case of the Airforce pilot because you’re unfit, then it is simply selfish.

It’s not the passengers fault, or the tower’s, or some Generals.

My concern on the other hand is with everyone else.

We can argue for days and not get any further.

I sent some mails and voiced my concerns. Let’s see if someone in charge of things like this will agree and have a closer look at what the kids in the AI labs do all day with their toys.

Thank you for the exchange and stay a decent person.

Oh, I agree it's a severe flaw of character that makes one unfit for duty. But flawed humans are what we have to work with! If the airline industry had a proper safety net around disclosing mental illness, we'd have fewer pilot murder-suicides. Are pilots that would do that severely flawed and unfit, maybe even evil? Definitely.

But our objective is to save lives, and to do that we have to identify those pilots, and to identify those pilots they have to be comfortable disclosing.

Essentially, I agree with you that these humans are deeply flawed, and my concern is also with everyone else. All I am saying is that more lives are saved with disclosure versus without. Avoiding punitive measures may let people who "don't deserve it" get away, but the disclosure you get in return saves hundreds to thousands more innocents.

i have given my reply elsewhere in this thread.

it covers the singificant problems in the approach of blaming a "mistake"; something that glosses over responsibility and doing wrong.

Reply on news.ycombinator.com