Hacker News Reader

Comment by xena | Hacker News Reader

Please don't use large language models as a therapist.

Religion and cults are next in the ML rabbit hole of content creation.

Time to re-watch the Terminator franchise and Matrix before reading up on the Butlerian Jihad, I guess. As the saying goes: Proper Planning Prevents Piss Poor Performance...

the people who create LLMs are just as psychotic as the ones who start cults, they just lack the je ne sais quoi to hypnotize people in person.

Your comment will probably get flagged but I have to admit I’m legitimately curious to find out if this holds any water.

snarky + opinionated !== flagged. it's okay to have a dark sense of humor; the mods aren't AIs. Yet.

I like your "about" - So what got you hell banned for 8 years?

heh. yeah, I didn't realize it because sometimes I'd still get responses from one or two people who could still see my posts. Definitely slowed me down after a few years of shouting into the void. Paul Graham did me a favor, really.

[edit] sorry, slightly dyslexic. so what got you was the question? Insulting Paul Graham.

I know the sorts of people who train LLMs, especially the ones who release them for free. They are the opposite of charismatic. You think ElutherAI is going to start a cult?

Already seen a guy lose his mind over supposed coincidences from ChatGPT that lined up with his cult fantasy.

ah now this...this is a brilliant man in the middle attack. Did you read that article by the girl who filled in holes for a chatbot for a year?

https://www.theguardian.com/technology/2022/dec/13/becoming-...

imagine if you hijacked, idk, some military general's ChatGPT session

[edit] Please tell the story about the guy who lost his mind over GPT coincidences, if you have the time to share it..?

Yeah perhaps just like this article.

This guy I knew was convinced that there were AI dopplegangers sent to replace the real versions of everyone. He thought somehow that I was a government agent after him even though we hadn’t spoken in years. Turns out though his supposed doppelgänger was a real dude who worked for the AI company that makes the AI personas. It gets better: when this person confronted his supposed doppelgänger, he responded by acting as if he were an actual AI, to troll the guy.

I won’t get into the cult stuff as I didn’t even try to keep up with it (maybe in my younger life) but he had a bunch of that as part of the conspiracy theory. Poor guy.

Wow, that's some real Philip K Dick territory. Hope the guy got help.

If you already have some idea of what you're looking for / in need of (as in talked to a therapist in the past), these language models can be a very good workbook, sounding board, self discussion tool.

> self discussion tool.

Self discussion as in "discussion between you and god knows who will get the data in the end" ?

Well yeah obviously read the TOS part about not sharing sensitive information.. in that sense it's not appropriate for therapy. But I've found it useful just keeping things general and not specific to details of my own life.

They work better than seven human ones I tried. Might just be my luck!

I don't doubt that experience, and I also have the comparison between humans and GPT and see where you are coming from. However, the advice still stands, in my eyes. If we recommend to check even simple python for correctness, how much more should we tell people to not trust LLMs when it comes to much more complex and influential things like therapy?

Also, you and some others might be able to understand the limitations of LLMs. It is the general public that I am worried about.

Yes, it is extremely limited, albeit in different ways than humans are. My point was just that as a profession, they aren't currently setting a very high bar.

There are always exceptional individuals, but sadly exceptional individuals can't scale to meet the needs of a society. (There's a serious shortage of therapists, teachers, etc., even counting all the lousy ones!)

> My point was just that as a profession, they aren't currently setting a very high bar.

I think I know where you are going, but I will be going offline for today soon, so I will be just leaving some thoughts regarding the matter. My background: Masters in CS, with a minor in Psychology. I have a psychology-ish circle of friends, some doing research, some doing therapy. I also have my fair share of negative experience with therapists due to some... uncommon combination of phenomena that require attention.

- One common mistake is confusing psychology (along with the more or less recent reproducibility crisis) with therapy (which is a profession that aims to help people, with some methods being grounded in psychology, some partially so, some other not at all).

- It would be weird for me to not acknowledge that there are some bad apples amongst therapists and how much it can suck being treated by one. Then again, I do not thing that the field of software development is any less diverse in the quality of output we deliver. Identifying software engineer that fulfils ones need is almost an art, maybe quite similar to finding a suitable therapist.

- "Finding a suitable therapist should not be an art!" Yes, would it not be nicer if everyone instantly found a suitable therapist? It would, but that is not the reality we live in. If even finding someone who can code a decent program that meets your relatively well-defined business needs, how much harder is it to find someone who treats multiple humans, with their unique problems, biographies and current lives?

- I am totally with you about the scaling issue! I am not against E-therapy and the usage of AI in it, but I think it has to be done with caution - not out of despair.

I am building a coaching journal, could be a therapist if you change the prompting.

The AI doesn’t have something else in their mind, a limited amount of time/energy for you.

Like radiologists and doctors the AI will be better than most of them really quick if not already.

Given the difference in cost, I can't picture a future in which this doesn't happen.

return to tradition, use

  emacs -nw -Q —-eval “(doctor)”

as your ancestors did!

TIL: Eliza lives in Emacs

https://en.wikipedia.org/wiki/ELIZA

  * Changes in Emacs 17

  ** Frustrated?

  Try M-x doctor.

  ** Bored?

  Try M-x hanoi.

  ** Brain-damaged?

  Try M-x yow.

  ** Sun3, Tahoe, Apollo, HP9000s300, Celerity, NCR Tower 32,
    Sequent, Stride, Encore, Plexus and AT&T 7300 machines supported.

  The Tahoe, Sun3, Sequent and Celerity use 4.2.  In regard to the
  Apollo, see the file APOLLO in this directory.  NCR Tower32,
  HP9000s300, Stride and Nu run forms of System V.  System V rel 2 also
  works on Vaxes now.  See etc/MACHINES.

and has done since ~1985!

Replika used GPT-3 in 2020 and it was marketed under the health section of the Google Play store.

Huggingface started as an attempt to use this stuff for therapy work. Hence their name. It's telling that they pivoted away from this verticle very quickly though!

Huh. I thought it was an 'Alien' reference.

Would be very interesting to see whether they do better or worse than the average human therapist. My own experience suggests it might be a closer than you suspect.

Really delusional to think you need to protect people from themselves. I don't get why HN puts therapy on such a pedistal.

Now I want to use a language model as a therapist.

why

Reply on news.ycombinator.com