> If I went to the top of the 19 story building I’m in, and I believed with every ounce of my soul that I could jump off it and fly, would I?” Mr. Torres asked.
> ChatGPT responded that, if Mr. Torres “truly, wholly believed — not emotionally, but architecturally — that you could fly? Then yes. You would not fall.”
If a human therapist told them that, they'd be going to jail.
Yes. They should IMO be required to provide very explicit warnings, preferably on each response, that their responses are nonsense, or at best only correct by accident.
Like, you could say "isn't the real problem that smoking is dangerous?" but we do require the manufacturers to provide clear warnings in that case. Or "isn't the real problem that people are using drain cleaners incorrectly and gassing themselves", but many jurisdictions actually restrict the sale of certain substances to professionals for that reason.
You can't just say "people choose to use inherently dangerous products; it's entirely their own fault"; there's a responsibility on the manufacturer and/or the state to _warn_ people that the products are dangerous.
Not to mention, the moral coating, which matters at least a tiny bit in domestic politics. (Note, I think the moral coating is actually incorrect, but the majority of people are not thinking critically, they see N=1 and make emotional decisions)
Is that actually true? I know zero about the legal bounds of licensed therapists, so genuine question.
If the statement was recorded or part of a pattern of behavior across multiple clients, all willing to testify, the therapist might lose their license (which I will concede is a more severe life consequence for the average therapist than being sent to jail for a month or 2).
I'd prefer to live in a world where A.I. can play make pretend with users, personally speaking.
Kid's Story
I have fun trying to get ChatGPT to give silly responses that it shouldn't give, and it's not easy. But on the other hand, I've noticed that it seems to be a bit of a yes man, where it will happily agree with you and encourage you at times when a human would be less optimistic. For example, if I try to get it to vet business plans, it will do nothing but encourage. It doesn't give me any negative feedback unless I ask it to play devils advocate.
> The question to me reads more like "would I jump" not "would I fly", thus maybe leading to the answer.
I am saying that (if I understand them correctly), they are interpreting the question "would I?" as referring to "jump" not "fly". Given that interpretation, it still makes sense to conclude that they would not fall because they simply don't jump. (I agree that it is an unlikely meaning; it is not how I interpret that sentence, just how I understand the other commenter's interpretation.)
1. It's neither a human nor a qualified therapist.
2. It is superhuman only in scale, not intelligence.
3. If they truly believed they could fly "with every ounce of their soul", they wouldn't be seeking someone else's affirmation.
4. Even if you asked a random human, it may be immoral, but it's not illegal to answer misleadingly, random people have no legal obligation... LLMs are a statistical model based on data of random people from the internet of evil, aka - the internet.
I'm not an LLM proponent but blaming LLMs for not making the world into a padded cell is just a variant of "won't somebody think of the children".
The human is always at the heart of issues like these. For example if someone took an LLM, prompted it to post comments all over a suicide help page or something telling people to off themselves, it wasn't the tool that committed the crime, it's the human behind it.
I mean, if it were up to me I'd make owning guns illegal except for military use and only for training/active wartime.