I have to wonder: would the suicide have been prevented if chatGPT didn't exist?
Because if that's not at least a "maybe", I feel like chatGPT did provide comfort in a dire situation here.
Probably we have no way not at least saying "maybe", but I can imagine just as well, that chatGPT did not accelerate anything.
I wished we could see a fuller transcript.
That's a pretty concerning take. You can provide comfort to someone who is despondent, and you can do it in a way that doesn't steer them closer to ending their life. That takes training though, and it's not something these models are anywhere close to being able to handle.
Maybe in the end ChatGPT would be a great tool to actually escalate on detecting a risk (instead of an untrue and harmful text snippet and a phone number).
Instead, we put the unlicensed therapist in jail.
I'd call the cops on them* at some point to stop them from harming themselves and I'd never say what ChatGPT said here, but I'd still talk to them trying to help, even without being a therapist. I can recommend a therapist, but it's hard to reach people in that state. You got to make use of the trust they gave you.
* non US country
I'd say yes, because the signs would have to surface somewhere else, probably in an interaction with a human, who (un)consciously saved him with a simple gesture.
With a simple discussion, an alternative perspective on a problem, or a sidekick who can support someone for a day or two, many lives can and do change.
We're generally not aware though.