A mental health tech company ran an AI experiment on real users | Hacker News Reader

A mental health tech company ran an AI experiment on real users

nbcnews.com

44 pointsdodgermax37 comments

> Nothing’s stopping apps from conducting more.

L-I-A-B-I-L-I-T-Y

Maybe that's just me, but if I was running a business that let's ChatBots impersonate my staff, I couldn't sleep at night thinking about scenarios, where my support chatbot starts offering steep discounts, refunds or even worse like in that particular case, suggests suic** to a patient with bad mental health.

You do realize that censoring a word in such a way as to not hide what the term is makes no difference? Not to single you out or anything, I see it a lot, but I am curious what you think it accomplishes.

Edit: Rephrased more politely.

For the same reason people online don't write "sex". To bypass overzealous contentfilters but keep it humanly readable.

Yeah many games now will block words even containing 'ass'. And they just do a complete substring search, so any of the totally normal words that happen to contain a s s are censored. I don't know why, but its so common that people will just censor themselves preemptively

I once read an article (probably on here) where a developer started tokenizing language into phonemes and then matching those against a list of verboten sounds. This prevented people (students?) from getting around literal filters. Basically hashing for English, where words like phat and 1337 could be matched. It always seemed like such an elegant and effective solution to a human vs computer problem. It would make a great library.

Why do we need to prevent people from expressing themselves? Why do we need to encourage such self-censorship as evident here? Why would the existence of such a library be a good thing?

Language is contextual. The use of certain terms is rude in some contexts, not in others. An algorithm will never be able to distinguish the two without being an AGI.

The only thing this encourages is self-censorship and dampening of expression.

The irony is that slurs gain their potency from their novelty which is emboldened by censorship and hindered by exposure. If taboo language was used freely and often, it would lose its shock value and subsequently its power to harm people sensitive to it.

I see it most often in YouTube videos where they’ll demonetize your video if you use any “bad” words.

Previously discussed:

Startup Uses GPT-3 to Provide Counseling and Then Realizes It 'Feels Weird'

https://news.ycombinator.com/item?id=34343928 (4 days ago, 7 comments)

We provided mental health support to about 4k people – using GPT-3

https://news.ycombinator.com/item?id=34294471 (8 days ago, 5 comments)

GPT-3 used to “provide” mental health support

https://news.ycombinator.com/item?id=34289096 (8 days ago, 2 comments)

Chatgpt would have been a better therapist for me than the online therapist I had a few years ago. I really felt like I was talking to a bot that didn’t parse the things I was saying.

I may be in the minority here, but I’d rather have chatgpt than a severely underpaid therapist with too many clients.

The human connection aspect of therapy is one of the primary reasons therapy works for me. Trust is a huge factor, and trust can only be built by establishing a relationship.

Trust isn’t just about “I can tell this person anything”, but also trust that the therapist understands me well enough to ask me the right questions, contextual to my experience in life, and in therapy. Trust that they are looking at my facial expressions and are noticing when I’m in distress even if I don’t realize I am in that moment. Trust that the person I’m speaking to has dedicated their life to help/support others, and that the things I say aren’t all being fed to some system that is analyzing every word.

Obviously YMMV, and this depends on why you’re in therapy, but one of the things that I’m deeply grateful for in therapy is that I know the person I’m speaking to deeply cares.

Maybe a bot can someday fill in for those times when you just need to have a conversation with someone other than yourself, or want someone to identify and help you see the thinking traps you’re falling into.

But I can’t help but feel such a bot should not be thought of as a therapist, any more than Tesla’s “Full Self Driving” feature should be considered fully autonomous. I think it’s a misnomer.

Trust is dead after what happened.

Trust in that specific product? No doubt, but that has little to do with the question of whether I’d rather speak with ChatGPT or a real therapist.

I'd rather have ChatGPT than a severely overworked/underpaid therapist, but I'd rather have a friend's shoulder to cry on than ChatGPT. But I'd also rather have a smart, empathic therapist I click with than oversharing with my friends.

Therapy for me is like, a psychic connection. I take someone deep into my headspace and they help me sort things out, exploring the inside of my mind. ChatGPT just makes that a loopback, a mirror that lets me see my mind in a new light sometimes. A real, useful therapist.. it's not scientific, but there's a connection I feel, and the possibility of emotions and thoughts being transmitted across. a human connection.

> it's not scientific, but there's a connection I feel, and the possibility of emotions and thoughts being transmitted across.

And I think it’s worth mentioning that “not scientific” just means science does not currently have much to say about the mechanisms in play, which is meaningful difference from a conclusion that there’s anything “woo woo” going on.

Mirror neurons are fascinating.

Do you prefer ChatGPD or the idea of ChatGPD?

It's one thing to like the idea of a super A.I. that does therapy, but I don't think ChatGPD can do basic things like ask you how your progress has been since last week's session.

There are two dimensions here. Good vs bad therapists, and therapist that can get through to you vs not.

Good vs bad you can solve with properly curated ratings from clients.

How to get through to you seems a harder problem. Sometimes it clicks and sometimes it’s a thud.

Regarding your second dimension, the "good" thing about ChatGPT is that you can request a new answer if the previous one doesn't vibe with you... Also, you could probably create infinite therapist personas that are tailored to your specific needs.

I don't know why'd you claim ChatGPD can create infinite therapists.

I just tried asking it to be a tough guy ex gang member therapist from the bronx who became a therapist and while it introduced itself that way it quickly dropped the personality and recited generic therapy advise.

I told it I was afraid of spiders. After it gave some advice and dropped the bronx character I asked it "Have you ever been scared of anything?"

And it said "As a machine learning model, I do not have personal experiences or emotions." Which shows it is not flexible.

It shows that you bounced off one of the artificial rails openAI added to minimize PR risk. The underlying model is certainly flexible enough to answer that question, it's just been deliberately crippled to try and make it seem less anthropomorphic.

But it broke character before it hit that rail and started talking like a magazine article, not a ex gang member turned therapist from the Bronx.

I mean, at this point why bother interacting with other people at all? Let's just have a collective of a few hundred chatGpt bots and pretend we're all a society.

I am horrified of how easily we are dismissing things that are super hard because "ai can do it better". Therapy is one of those things were result vary widely and depend on the patient, the therapist and the setting. You ain't gonna get that from a machine - not with the way we are doing AI today.

I’d much rather wait for an AI solution. To wit, I would rather tweak a set of configuration options than go through the process of dealing with the mental health industry. It is overpriced, unreliable, and unworthy of the trust that it requires. I would sooner trust a machine.

Yes and no. You're assuming that everyone has the mental and physical ability to deal with something like this. Most people are not technical enough for really really basic tech tasks. It is rare that you would find someone who is a good therapist and can write software. Even rarer that someone would actually invest the time to build you the software you could tweak.

I believe in the singularity, so I guess I meant to say AGI. Such an AI will learn to be a therapist the same way that we do. Even if it’s only as smart as a human, I would trust it more than a human.

Because, yeah, I don’t trust humans to program an AI that I could ever trust, unless the model and training data are all made fully available for independent analysis.

Tweaking the model would be telling it to favor certain paradigms over others.

If (and it's a huge if) we ever get AGI we're either 1) screwed and the machines will remeber us like we remember cave people - ie footnote in the history "books" or 2) we will have a human + machine symbiosis where the AI will literally remove all out flaws.

Either way, therapy will be useless. Also, IMHO we are at least 500 years from anything that resembles AGI - if it ever happens.

Yes.

OpenAI is very upfront and in your face with its heads-up regarding the lack of privacy of ChatGPT sessions. (You can be identified from your writing style.)

Beyond mere privacy issues, the fact that they would deceive humans with mental health issues, lulling them into a state of suspended disbelief regarding the competence and credentials of the 'therapist', and exposing them to potentially harmful feedback appears to be (ianal) a criminal act.

Whoever is doing this deserves to get sued to oblivion.

"A chat app used for emotional support used a popular chatbot to write answers for humans to select"

“Real users” is rather sterile when users were asking for mental health assistance in the form of emotional support.

Trying ChatGPT on a user base isn’t inherently bad. Doing so without their knowledge isn’t inherently bad.

This just blew past several sanity checks. Among other things.

I’d beg to differ.. if we’re talking about customer support, it wouldn’t be that interesting. But closer to the realm of mental health, the lack of notice qualifies as concerning in my book. Given the error rate of ChatGPT, it also not assured that it wouldn’t respond with something inappropriate or dangerous.

What part of “blew past several sanity checks” gave you the impression I’m not concerned?

> Doing so without their knowledge isn’t inherently bad.

Appears to carry more weight than "sanity checks". IME, the latter are often just advisory to catch cases that would be 'insane'.

“Isn’t inherently bad” together with “just” are pretty minimizing

From TFA:

- Users were notified that the posts were written in collaboration with a bot

- Human authors oversaw the posts, including having the ability to edit posts and having responsibility for hitting send.

The lack of informed consent from study participants would likely have killed this experiment in a normal IRB review, but they didn't just sic ChatGPT loose on people seeking psychological help.

LOL @ NBC ad-walling the article. Reminds me of when CNN wanted people to pay $20/month to watch videos on cnn.com.

Anyway I was wondering if this was the Aura-Lima Jevremovic-Amanda-Soft White Underbelly saga. I guess not.

Reply on news.ycombinator.com