'It cannot provide nuance': UK experts warn AI therapy chatbots are not safe
theguardian.com
theguardian.com
Edit: the study compared therapist outcomes to AI outcomes to placebo outcomes. Therapists in this field performed slightly better than placebo, which is pretty terrible. The AI outcomes performed much worse than placebo which is very terrible.
Some people knew what the tobacco companies were secretly doing, yet they kept quiet, and let countless family tragedies happen.
What are best channels for people with info to help halt the corruption, this time?
(The channels might be different than usual right now, with much of US federal being disrupted.)
And if the alleged payer is outside the field, this might also be relevant to the public interest in other regards. (For example, if they're trying to suppress this, what else are they trying to do. Even if it turns out the research is invalid.)
I agree. Asking questions which are normal in my own field resulted in stonewalling and obvious distress. The worst thing being this leading to the end of what was a good relationship.
If not, you might consider whether you have actionable information yourself, any professional obligations you have (e.g., if you work in science/health/safety yourself), any societal obligations, whether reporting the allegation would be betraying a trust, and what the calculus is there.
Also it is not expected that the training material for the model deals with the actual practical aspects of therapy, only some of the theoretical aspects are probably in that material
BrickLabs have developed an expert-fine-tuned model specifically to provide psychotherapy. Their model has shown modestly positive results in a reasonably large preregistered RCT.
waitlist control, where people get nothing
psychoeducational, where people get some kind of educational content about mental health but not therapy
existing nonpsychological service, like physical checkups with a nurse
existing therapy, so not placebo but current treatment
pharmacological placebo, where they're given a placebo pill and told its psychiatric medication for their concern
A kind of "nerfed" version of the therapy, such as supportive therapy where the clinician just provides empathy etc but nothing else
How to interpret results depends on the control.
It's relevant to debates about general vs specific effects in therapy (rapport, empathy, fit) versus specific effects (effects due to specific techniques of a specific therapy).
Bruce Wampold has written a lot about types of controls although he has a hard nonspecific/general effects take on therapy.
Otherwise, you may end up defending this and it's really foolish:
> “Seriously, good for you for standing up for yourself and taking control of your own life,” it reportedly responded to a user, who claimed they had stopped taking their medication and had left their family because they were “responsible for the radio signals coming in through the walls”.
Man I hate this modern shift of “actually anyone who is an expert is also trying to deceive me”. Extremely healthy shit for a civilization.
That said, I certainly don't see therapists as profiteering, in the sense of trying to convince people to pay for therapy they don't need. They might plausibly feel threatened by AI, but they'd absolutely be justified in calling out examples like those in TFA.
Hot take: Therapists should earn more than most software devs.
Ahh, here we go; Indeed.com includes "Stretch Therapist", Occupational Therapist, Respiratory Therapist, Weight Gain Therapist, Massage Therapist, Care Co-ordinator, and everything else that includes the term "Therapist" in it.
Assuming that anyone who has anything to gain by you believing them is out to get you is rash and leads you only to those who are more willing to lie about their motivations. Yes, doctors and Big Pharma (tm) are financially motivated to sell you cures, but the guy selling you a juice cleanse ""at cost"" for your cancer is still not trustworthy.
Two things can be true at the same time. Don't trust anyone because nobody is transparent about their incentives. Your doctor does not disclose to you that they were at a congress in Hawaii for company X when he prescribes you to use company X drug for your ailment.
If they warned that it could become a distillation of the worst aspects of television... maybe they weren't wrong.
Fuck me. Maybe that guy on the street corner selling salvation or “cuckane” really was dealing in the real thing, too, eh?
(But yeah, relying on systems that can have bugs like that for your mental health is terrifying.)
They didn't feel threatened by systems like cleverbot or GPT-3.5
Try this on for size: I am not a therapist, but I will happily tell you that a statistical word generating LLM is a truly atrocious substitute for the hard work of a creative, empathetic and caring human being.
Almost all of these people were openly in (romantic) love with these agents. This was in 2017 or thereabouts, so only a few years after Spike Jonze’s Her came out.
From what I understand the app is now primarily pornographic (a trajectory that a naiver, younger me never saw coming).
I mostly use Copilot for writing Python scripts, but I have had conversations with it. If the model was running locally on your own machine, I can see how it would be effective for people experiencing some sort of emotional crisis. Anyone using a Meta AI for therapy is going to learn the same hard lesson that the people who trusted 23 and Me are currently learning.
People really like to anthropomorphize any object with even the most basic communication capabilities and most people have no concept of the distance between parroting phrases and a full on human consciousness. In the 90s Furbys were a popular toy that said started off speaking furbish and then eventually spoke some (maybe 20?) human phrases, many people were absolutely convinced you could teach them to talk and learn like a human and that they had essentially bought a very intelligent pet. The NSA even banned them for a time because they thought they were recording and learning from surroundings despite that being completely untrue. Point being this is going to get much worse now that LLMs have gotten a whole lot better at mimicking human conversations and there is incentive for companies to overstate capabilities.
There are psychological blindspots that we all have as human beings, and when stimulus is structured in specific ways people lose their grip on reality, or rather more accurately, people have their grip on objective reality ripped away from them without them realizing it because these things operate on us subliminally (to a lesser or greater degree depending on the individual), and it mostly happens pre-perception with the victim none the wiser. They then effectively become slaves to the loudest monster, which is the AI speaking in their ear more than anyone else, and by extension to the slave master who programmed the AI.
One such blindspot is the consistency blindspot where someone may induce you to say something indicating agreement with something similar first, and then ask the question they really want to ask. Once you say something that's in agreement, and by extension something similar is asked, there is bleedover and you fight your own psychology later if you didn't have defenses to short circuit this fixed action pattern (i.e. and already know), and that's just a surface level blindspot that car salesman use all the time; there are much more subtle ones like distorted reflected appraisal which are used by cults, and nation states for thought reform.
To remain internally consistent, with distorted reflected appraisal, your psychology warps itself, and you as a person unravel. These things have been used in torture, but almost no one today is taught what the elements of torture are so they can recognize it, or know how it works. You would be surprised to find that these things are everywhere today, even in K12 education and that's not an accident.
Everyone has reflected appraisal because this is how we adopt the cultural identity we have as people from our parents while we are children.
All that's needed for torture to break someone down are the elements, structuring, and clustering.
Those elements are isolation, cognitive dissonance, coercion with perceived or real loss, and lack of agency to remove with these you break in a series of steps rational thought receding, involuntary hypnosis, and then psychological break (disassociation or a special semi-lucid psychosis capable of planning); with time and exposure.
Structuring uses diabolical structures to turn the psyche back on itself in a trauma loop, and clustering includes any multiples of these elements or structures within a short time period, as well as events that increase susceptibility such as narco-analysis/synthesis based in dopamine spikes triggered by associative priming (operant conditioning). Drug use makes one more susceptible as they found in the early 30s with barbituates, and its since been improved so you can induce this is in almost anyone with a phone.
No AI will ever be able to create and maintain a consistent reflected appraisal for the people they are interacting with, but because the harmful effects aren't seen immediately, people today have blinded themselves and discount the harms that naturally result. The harms from the unnatural loss of objective reality.
The coursework in an introduction to communication class may provide some foundational details (depending on the instructor), Sapir-Whorf has basis in blindspots.
Robert Lifton touches on the detailed case studies of torture from the 1950s (under Mao), in his book "Thought Reform and the Psychology of Totalism", and I've heard in later books he creates a framework that classifies cultures as Protean (self-direction, growth, self-determination/agency), or Totalism (towards control which eventually fails Darwin's fitness).
I haven't actually read his later books yet though his earlier books were quite detailed. I believe the internet archive has a copy of this available for reading as a pdf but be warned this is quite dark.
Joost Meerloo in his, "Rape of the Mind" as an overview touches on how Totalitarianism grows in the setting of WW2 and some Mao, though takes Freudian look at things (dating certain aspects which we know to be untrue now).
From there it branches out depending on your interest. The modern material itself while based on these earlier works often has the origins obscured following a separation of objectionable concerns.
There are congressional reports on COINTELPRO and you may find notice it has modern iterations (touching on protest/activist activity harassment), as well as the history of East German Stasi, and Zersetzung where governments use this to repress the population.
There are aspects in the Octalysis Framework (gamification/game design).
Paulo Freire used some of this material in developing his critical pedagogy which was used in the 70s to replace teaching method from a reduction of first principles (based in rome and the greeks) to what's commonly known as rote-based teaching, and later called "Lying to Children", which takes the reversal of that approach following more closely to gnosticism.
The approach is basically you give a flawed useless model which includes both true and false things. Students learn to competence, then are given a new model that's less flawed, where you have to learn and unlearn things already learned. You never actually unlearn anything and it induces frustration and torture destroying minds in the process. Each step towards gnosis becomes more useful but only the most compliant and blind make it to the end with few exceptions. Structures that burn bridges induce failure in math, and the effect is this acts as a filter to gatekeep the technical fields.
The water pipe analogy of voltage in electronics as an example of the latter instead of the first principled approach using diffusion which is more correct.
Disney and Dreamworks uses distorted reflected appraisal tailored towards destructive interference of identity, which some employees have blown the whistle on (for the latter), aimed at children and sneak things past their adult guardians. There's quite a lot if you look around but its not under any single name but scattered. Hopefully that helps.
The Dreamworks whistleblower interview can be found here: https://www.youtube.com/watch?v=vvNZRUtqqa8
All indexed references of it seem to now have been removed from search. I'm glad now that I kept a reference link in a text file.
Update: Dreamworks isn't Pixar, I misremembered,they are owned by Universal Studios, whereas Disney own's Pixar. Pixar and Disney appear to do the same things.
I’m not sure I understand how this relates to gnosticism, however. Are you comparing the “Lying to Children” model to gnostic initiation, and asserting that this model selects for the compliant? What is your proposed alternative here?
Particularly,
> Structures that burn bridges induce failure in math, and the effect is this acts as a filter to gatekeep the technical fields.
Sounds compelling, but it strikes me more as a limitation of demand for good math teachers outstripping their supply. I’ve seen this in English language learning a lot; even if the money was there (and it’s not), there are simply far more people with a desire to learn English than there are people qualified to teach it.
> It sounds like you're pointing to something more nefarious.
Well the structure itself is quite nefarious in a way. You have to constantly fight against it to progress and don't really have a choice at the beginning, which often leads to learned helplessness and PTSD in the dropouts. As a teacher you also have to constantly fight against this because any shortfall of effort on your part leaves your students behind in one of those pitfalls, and its largely dependent on the students ability to overcome the torture. You generally aren't given sufficient resources to do this because there's no way out; only through. This is why the structure is nefarious and at the root of the problem.
The unlearning process after learning to competence is imperfect and induces what amounts to self-torture sessions. The imposition of psychological stress (torture) actually lowers the ability for rational thought, and may permanently warp people at vulnerable stages of their lives. Children tend to have a period where they try on various personas after which their identity crystallizes which they carry forward. Adopting learned helplessness at this point makes them a resource drain on everyone. You see these effects in the youth today where they can't even read in many cases.
The sequences in math for example rely on a undisclosed change in grading criteria resulting from this path, a gimmick if you will. There is the sequence, Algebra->Geometry->Trigonometry. Algebra is graded based on correct process, whereas Trig is graded based on correct process and correct answer. When the process differs between classes because the process taught was a flawed version, and you pass Geometry, you can't go back. Its outside the scope of the Trig teacher to reteach two classes prior, and they'll just say: "If you are having trouble with this material you should choose a career that doesn't require this", and leave it up to them. This was actually pushed for adoption by the NEA in the 90s, where they were going to strike if the administration didn't cave.
There are similar structures used in weed-out classes in college as well. Physics used to use a non-standard significant figure calculation when the questions were related by a property of causality (1st answer is used for the 2nd, and the 2nd for 3rd, 2 tests, you can only get 1 question wrong on one test to pass. It must be either of the last two on either test). Using a correct method to reduce propagation of error would cause you to fail, and the right answer was passed around to only the professor's favorites, hence very similar to gnosticism where the only the experts determine who may receive the secret knowledge.
An excellent teacher that constantly bucks the norm will naturally sidestep many of the pitfalls, but an average teacher who is overburdened from lack of resources, and ground down who has sunk to the lowest common denominator of work production won't provide a bridge over the pitfall and these things happen through simple lack of action as a consequence of the adopted structure.
When people speak of nefarious and maliciousness there's often an assumed intent, and in a way negligence can be intent but while some could argue these type of plans conform to this based on things our nation's enemies have said, its probably equally if not more a result of degradation and corruption from within as a result of the flaws inherent in centralized systems.
The history about how this came about is particularly muddied. To give some context, Sputnik in the 1960s shocked the US, and they wrote a blank check for Academia towards more engineers and math alumni. It was a problem you can't fix though using money, and when that was noticed the hiring standards which were quite high in the 1960s, were lowered. Whether the lower standards caused this, or subversives snuck in as an attack on the next generation, no one will know. The effect though is by 1978 there is a marked difference in the academic material published prior and after with lower quality resources being available after which conform to the mentioned flawed pedagogy.
The proposed alternative is to go back to the classical pedagogical approach. Use real systems, teach the process of reducing those systems to first principles (in guided fashion), creating models, and then predicting the future behavior of those systems, identifying the limitations. Some professors still do this, but they are in such a minority that you may only see on or two in a local geography (driving range/county) across all areas of study.
> Sounds compelling but it strikes me more as a limitation of demand for good math teachers.
I've known quite a lot of extremely intelligent people who have been hobbled because they couldn't get through the education, the few that have are often unable to apply the knowledge outside a very limited scope. Its a bit of a chicken egg problem, you need the chicken first.
The hiring standards were never raised back up and remain low, and the materials used to teach those have degraded, there is also no incentive towards improvement of teachers. Basic performance metrics are eschewed from collection. You see this particularly in colleges where they may collect pass rates but won't differentiate a person who has taken the class in the past from a new student.
There are also other incentives which are covered quite plainly in the documentary "Waiting for Superman" in the Lemon walk. If you don't fire your lowest performers, and they are effectively guaranteed wages without the appropriate level of work, they end up driving the higher performers out through social coercion, harassment, and corruption. The higher performers make the lower performers look bad.
What you are referencing is more along the lines of ideology, which can act as an indirection. Ideology generally lacks properties of metaphysical objectivity whereas what I'm referencing has actually happened (objectively and in reality).
> It sounds really Althusserian ...
What has happened does follow quite closely to various branches of Marxism, which does make a rational person question if this wasn't intended subversively as an attack, but as I think I mentioned before there's no proof whether it was the reduction of hiring standards, or a subversive cohort effort seeking to undermine the next generation. The simple fact is, when you look at the structure of either, they are effectively the same in that they lead to the same outcomes. They suffer from the same structural flaws so supporting one over the other would only be about supporting whether there was intent, general or specific, the former might be met by negligence. Whatever happens to be the case, it is immaterial to the outcomes which we've seen.
In either case it leads to destruction/annihilation through a process that results in insanity, which is itself quite insane when you consider we live under MAD doctrine, and existentially we, and everyone else, only exist today because the level heads of rational leaders have managed to not engage, and de-escalate close calls related to thermonuclear war.
What becomes likely to happen when you've sufficiently induced psychosis in the people whose hands can push that button seems quite predictable... You don't want insane people controlling your ability to exist.
Given the existential threat, this is why this type of attack would be unthinkable and insanity, and by extension why many of these ideologies based on systems that don't work are often referred to as death cults.
Its not that the sentiment towards providing necessity, and reducing abuse is itself not something to try to make happen, but it has to happen rationally based in external reality and not blindly, the latter which results in those destructive outcomes when you don't recognize the underlying principles and laws which those outcomes are based in.
In many respects, we as a society are in crisis.
The difference between what is said, and what we know to be true has become an abyss. Of all risks, the loss of an objective reality is perhaps the most dangerous, and the death of truth is the ultimate victory of evil; which is the loudest monster that will destroy us all.
This is why it is so important to go back to what we know worked. A Classical trivium based education worked following the pedagogy of Method and reduction to first principled approaches.
The reality of these changes, regardless of how they came about, is that this has dramatically degraded the general populace's ability to think, recognize, comprehend, and solve problems. Its directly tied to education which has been based in torture, and which even the teachers themselves don't seem to have realized.
Western society has been based in a culture of proteanism, rational thought, self-determination, agency, and growth.
What's being indoctrinated is a culture of totalism; total control without flexibility. The latter type of society only ever leads to eventual extinction when they become unable to adapt to the changing circumstances of reality.
The world would like quite different if this was true.
He seems so desperate to sell AI that he forgot such thing already exists. It's called family or a close friend.
I know there are people who truly have no one and they could benefit from a therapist. Having them rely on AI could prove risky specially if the person is suffering from depression. What if AI pushes them towards committing suicide? And I'll probably be told that OpenAI or Meta or MS can put guardrails against this. What happens when that fails (and we've seen it fail)? Who'll be held accountable? Does an LLM take the hippocratic oath? Are we actually abandoning all standards in favour of Mark Zuckerberg making more billions of dollars?
It's good you are socially privileged, but a lot of people do not have someone close who they can feel secure to confide in. Even a therapist doesn't help here, as a lot of people have pre-existing conditionings about what a therapist is "I'm not crazy, why do I need a therapist?".
Case in point, my father's cousin lived alone and didn't have any friends. He lived in the same house his whole life, just outside London by himself, with no indoor toilet or hot water. A few years ago, social services came after the neighbours called, because his roof collapsed and he was just living as if nothing was wrong. My father was his closest living family, but they'd not spoken in 20 years or more.
I feel this kind of thing is more common than you think. Especially with older people, they may have friends from the outside, but they aren't close with them that they can talk about whatever is on their mind.
What you described isn't a good fit for using AI. What would an LLM do for him?
The fact his roof collapsed and he didn't think much of it indicates a deeper problem only a human can begin to tackle.
We really shouldn't be solving deep societal problems by throwing more tech at them. That experiment has already failed.
The point being fixing your own life is going to bring much more in the way of benefits than the government or Sam trying to fix it for you. If one are a complete social reject then no amount of AGI will save them. People without close relationships are zombies that walk among us, in most ways they are already dead.
I 100% do not doubt the usefulness of therapy for those who are suffering in some way, but I feel like the idea that "everyone should probably have a therapist" is kinda odd - if you're generally in a good place, you can explore your feelings/motivations yourself with little risk.
> In a separate interview last week, Zuckerberg said “the average American has three friends, but has demand for 15” and AI could plug that gap.
And I think we should definitely look on this tech with scrutiny, but I think another angle to look at it is: which is worse? No therapy or AI therapy? You mention suicide, but which would result in a reduction in suicide attempts, a or b? I don't have an answer, but I could see it being possible that because AI therapy provides cheaper, more frequent access to mental care, even if it is lower quality, it could result in a net improvement over the status quo on something like suicide attempts.
1) Chatbots are never going to be perceived as safe or effective as humans by default, primarily due to human fiat. Professionals like counselors (and lawyers, doctors, software engineers, etc.) will always claim that an LLM cannot do their job, namely because acknowledging such threatens their livelihood. Determining whether LLMs genuinely provide therapeutic value to humans would require rigorous, carefully controlled experiments conducted over many years.
2) Chatbots definitely cannot replace human therapists in their current state. That much seems quite obvious to me for various reasons already argued well by others on here. But I had to highlight point #1 as devil's advocate, because adopting the mindset that "humans are inherently better by default" due to some magical or scientifically unjustifiable reason will prevent forward progress. The goal is to eliminate the (quite reasonable) fear people have of eventually losing their job to AI by enacting societal change now rather than denying into perpetuity that chatbots are necessarily inferior, at which point everyone will in fact lose their jobs because we had no plan in place.
It may be able to assist those professionals, but that is as far as I am willing to go, because I am not blinded by the shine of the statistical turks we are deploying right now.
Compared to that status quo, I'm not sure that LLMs are meaningfully more risky - unlike a human, at least it can't physically assault you.
https://www.bacp.co.uk/news/news-from-bacp/2020/6-march-gove...
https://www.theguardian.com/society/2024/oct/19/psychotherap...
For counselling, people are encouraged to choose counsellors accredited by professional orgs like BACP.
The BACP's standards really aren't very high, as you can qualify for membership after a one-year part-time course and a few weeks of work experience. Their disciplinary procedures are, in my opinion, almost entirely ineffectual. They undertake no meaningful monitoring of accredited members, relying solely on complaints from members of the public. Out of tens of thousands of registered members, only a single-digit number are subject to disciplinary action every year. The findings of the few disciplinary hearings they do actually conduct suggest to me that they are perfectly happy to allow lazy, feckless and incompetent practitioners to remain on their register, with only a perfunctory slap on the wrist.
BACP membership is of course entirely voluntary and in no way necessary in order to practice as a counsellor or psychotherapist.
https://www.hcpc-uk.org/news-and-events/blog/2023/understand...
https://www.bacp.co.uk/about-us/protecting-the-public/profes...
As a result, I agree with you.
It gives me pause when I stop to think about anyone without more context placing so much trust in these. And the developers engaged in the “industry” of it demanding blind faith and full payment.
We may just need to start comparing success rates and liability concerns. It's kind of like deciding when unassisted driving is 'good enough'.
AI is not a substitute for traditional therapy, but it offers an 80% benefit at a fraction of the cost. It could be used to supplement therapy, for the periods between sessions.
The biggest risk is with privacy. Meta could not be trusted knowing what you’re going to wear or eat. Now imagine them knowing your deepest darkest secrets. The advertising business model does not gel well with providing mental health support. Subscription (with privacy guarantees) is the way to go.
Therapy seems like the last place an LLM would be beneficial because it’s very hard to keep an LLM from telling you what you want to hear. I can see anyway you could guarantee that a chatbot cause severe damage to a vulnerable patient by supporting their neurosis.
We’re not anywhere close to an LLM which is trained to be supportive and understanding in tone but will never affirm your irrational fears, insecurities, and delusions.
When an AI therapist from a health startup confirms that a mentally disturbed person is indeed hearing voices from God, or an insecure teenager uses meta AI as a therapist because Mark Zuckerberg said they should and it agrees with them that yes they are unloveable, then we have a problem.
There's also talking to a friend. Sure, they could also steer you wrong, but at least they won't be impersonating a therapist, and they won't be doing it to try to please their investors.
They will not under any circumstances tell you that “yes you are correct, Billy would be more likely to love you if you drop 30 more pounds by throwing up after eating”, but an LLM will if it goes off script.
This is an implementation problem and not really a technical limitation. If anything, by focusing on a particular domain (like therapy), the do’s and don’ts become more clear.
There is a very fine line between being understanding and supportive and enabling bad behavior. I’m not confident that a team of LLMs is going to be able to walk that line consistently anytime soon.
We can’t even get code generating LLMs to stop hallucinating APIs and code is a much narrower domain than therapy.
It’s been incredibly helpful for my personal use: brainstorming ideas, such as exploring how different scenarios might unfold. For instance, I can ask, “What are the pros and cons of choosing x over y, considering these factors?” or even, “I’m in a tough spot. X and I often argue about Z (provide some background context), and I’m struggling to express my perspective. I’m afraid…” You get the idea.
GPT-4o is remarkably good at putting things in an independent and unbiased third person perspective. It’s definitely not an echo chamber for me. More often, the insights are what I might have come up with if I were observing my own life from a distance.
Now some people have said “sure it’s like journaling”. I think it’s even better, like talking to your journal (ala Harry Potter, like Tom Riddle’s diary) with some level of fact checking (I’ve gotten called out) and human behavioural understanding available at your disposable.
And it’s not just code where they go off the rails. If you talk to them for a while they will very frequently end up agreeing with you if you want them to.
I’ve seen this many times when using an LLM to try to learn something new or refresh my memory.
How can you so confidently claim that "Therapists will do this and that, they won't do any evil". Did you even read what you posted?
This is very untrue. Here is a list of psychotherapy modalities: https://en.wikipedia.org/wiki/List_of_psychotherapies. In most (almost every) modalities, the therapist provides an intervention and offers advice (by definition: guidance, recommendations).
There is Carl Rogers' client-centered therapy, non-directive supportive therapy, and that's it for low-intervention modalities off the top of my head. Two out of over a hundred. Hardly "most" at all.
https://en.m.wikipedia.org/wiki/Person-centered_therapy
That sounds an awful lot like what current gen AIs are capable of.
I believe we are in the very early stages of AI-assisted therapy, much like the early days of psychology itself. Before we understood what was generally acceptable and what was not, it was a Wild West with medical practitioners employing harmful techniques such as lobotomy.
Because there are no standards on what constitutes an emotional support AI, or any agreed upon expectations from them, we can only go by what it seems to be capable of. And it seems to be capable of talking intelligently and logically with deep empathy. A rubber ducky 2.0 that can organize your thoughts and even infer meaning from them on demand.
That... seems optimistic. See, for instance, https://www.rollingstone.com/culture/culture-features/ai-spi...
No psychologist will attempt to convince you that you are the messiah. In at least some cases, our robot overlords are doing _serious active harm_ which the subject would be unlikely to suffer in their absence. LLM therapists are rather likely to be worse than nothing, particularly given their tendency to be overly agreeable.
I'm sure 80% of expert therapists in any modality will disagree.
At best, AI can compete with telehealth therapy, which is known for having practically no quality standards. And of course, LLMs surpass "no quality standards" with flying colors.
I say this very rarely because I think such statements should be used with caution, but in this case: saying that LLMs can do 80% of a therapist's work is actually harmful for people who might believe it and not seek effective therapy. Going down this path has a good probability of costing someone dearly.
Given that, AI can be just as good as talking to a friend when you don’t have one (or feel uncomfortable discussing something with one).
This is not true, and it's not even wrong. You almost cannot argue with such a statement without being ridiculous. The best I can say is: natural language synthesis is not a substitute for friends.
If we are debating these things, it's evidence we adopted LLMs with far too little forethought.
I mean, on a technicality, you could say "my friend synthesizes plausible language, this can do it, too. So it can substitute a little bit!" but at that point I'm pretty sure we're not discussing friendship in its essence, and the (emotional, physical, social, etc) support that comes with it.
“A friend” can also serve as a metaphor for an acquaintance you feel comfortable seeking counsel from.
This sentence effectively reads "AI cam be just as good as (nothing)" since you can't talk to a friend when you don't have one.
Of course, I understand the point you were trying to make, which is that AI is better than absolutely nothing; but I disagree in the vain that AI will give you a false since of companionship that might lead you further towards bad outcomes.
No, the biggest risk is that it behaves in ways that actively harm users in a fragile emotional state, whether by enabling or pushing them into dangerous behavior.
Many people are already demonstrably unable to handle normal AI chatbots in a healthy manner. A "therapist" substitute that takes a position of authority as a counselor ramps that danger up drastically.
Also, for every nay sayer I encounter now, I’m going to start by asking “Have you ever taken therapy? For how long? Why did you stop? Did it help?”
Therapy isn’t a silver bullet. Finding a therapist that works for you takes years of patient trial and error.
the problem is that "responsible deployment" feels extremely at odds with, say, needing to justify a $300B valuation
This part is what I take issue with. Software is simply different from buildings.
The average person will never have the required experience to make an informed decision on the efficacy and safety of this.
This is something that bugs me about medical ethics, that it's more important not to cause any harm than it is to prevent any.
That of course doesn't exclude doing good, being helpful, using skills and technologies to produce favorable outcomes. It does mean that healers must exercise due vigilance for unintended adverse consequences of therapies, let alone knowingly providing services that cause harm.
The problem with "safe/not safe" designation is simply that these states are more often than not indistinct. Or put another way, it depends on subtle contextual attributes that are hard to discern. Furthermore individual differences can make it difficult to predict safety of applying a procedure.
As a result healers should be cautious in approaching problems. Definitely prevention is better than cure, it's simply that relatively little is known about preventing burdensome conditions. Exercising what is known is a high priority.
I think GP understands this, and disagrees with the principle.
I don't know that AI "advisory" chatbots can replace humans.
Could they help an individual organize their thoughts for more productive time with professionals? Probably.
Could such tech help individuals learn about different terminology, their usage and how to think about it? Probably.
Could there be .. a net results of spending fewer hours (and cost if the case) for the same progress? And be able to make it further with advice into improvement?
Maybe the baseline of advisory expertise in any field exists more around the beginner stage than not.
Experience matters, that's something we seem to be forgetting fast.
Which begs the question, why do so many people currently need therapy? Is it social media? Economic despair? Or a combination of factors?
We've also stigmatized a lot of the things that folks previously used to cope (tobacco, alcohol), and have loosened our stigma on mental health and the management thereof.
I'd disagree. If you worked in the fields, you have plenty of time to think. We fill out every waking hour of our day, leaving no time to ponder or reflect. Many can't even find time to workout and if they do they listen to a podcast during their workout. That's why so many ideas come to us in the shower, it's the only place left where we don't fill out minds with impressions.
It's just so much easier to externalize everything and constantly be looking to your environment and how it influences your life, as opposed to looking within. It's very uncomfortable to try to figure out why you are the way that you are and what you can do about it.
There's so much history that shows that people have always been able to think like this, and so much written proof that they have, and to the same proportion as they do today.
Besides, in 12 hour days on a field, do you not have another 4 hours to relax and think? While stalking prey for 5 miles, is it not quiet enough for you to reflect on what you're doing and why?
I do think you're onto something though when you say it's related to our material needs all being relatively met. It seems that's correlational and maybe causal.
Actually, around here, you are lucky to find a job that is NOT 12 hours a shift.
What I notice is that the old members keep the younger members engaged socially, teach them skills and give them access to their extensive network of friends, family, previous (or current) co-workers, bosses, managers. They give advise, teach how to behave and so on. The younger members help out with moving, help with technology, call an ISP, drive others home, to the hospital and help maintain the facilities.
Regardless of age, there's always some dude you can talk to, or knows who you need to talk to, and sometimes there's even someone who knows how to make your problems go away or take you in if need by.
A former colleague had something similar, a complete ready so go support network in his old-boys football team. Ready to support in anyway they could, when he started his own software company.
The problem: This is something like 250 guys. What about the rest? Everyone needs a support network, if your alone, or your family isn't the best, you only have a few superficial friends, if any, then where do you go? Maybe the people around you aren't equipped to help you with your problems, not everyone is, some have their own issues. The safe spaces are mostly gone.
We can't even start up support networks, because the strongest have no reason to go, so we risk creating networks of people dragging each other down. The sports clubs works because members are from a wider part of society.
From the article:
> > Meta said its AIs carry a disclaimer that “indicates the responses are generated by AI to help people understand their limitations”.
That's a problem, because most likely to turn to an LLM for mental support don't understand the limitations. They need strong people to support and guide them, and maybe tell them that talking to a probability engine isn't the smartest choice, and take them on a walk instead.
1. The effects of AI should not be compared with traditional therapy, instead, they should be compared with receiving no therapy. There are many people who can't get therapy, for many reasons, mostly financial or familial (domestic abuse / controlling parents). Even for those who can get it, their therapist isn't infinitely flexible when it comes to time and usually requires appointments, which doesn't help with immediate problems like "my girlfriend just dumped me" or "my boss just berated me in front of my team for something I worked 16-hour days on."
AI will increase the amount of therapy that exists in the world, probably by orders of magnitude, just like the record player increased the amount of music listening or the jet plane increased the amount of intercontinental transportation.
The right questions to ask here are more like "how many suicides would an AI therapist prevent, compared to the number of suicides it would induce?", or "are all human therapists licensed in country / state X more competent than a good AI?"
2. When a person dies of suicide, their cause of death is, and will always be, listed as "suicide", not "AI overregulation leading to lack of access to therapy." In contrast, if somebody dies because of receiving bad AI advice, that advice will ultimately be attributed as the cause of their death. Statistics will be very misleading here and won't ever show the whole picture, because counting deaths caused by AI is inherently a lot easier than counting the deaths it prevented (or didn't prevent).
It is much safer for companies and governments to prohibit AI therapy, as then they won't have to deal with the lawsuits and the angry public demanding that they do something about the new problem. This is true even if AI is net beneficial because of the increased access to therapy.
3. Because of how AI models work, one model / company will handle many more patients than any single human therapist. This means you need to rethink how you punish mistakes. Even if you have a model that is 10x better than an average human, let's say 1 unnecessary suicide per 100000 patients instead of 1 per 10000, imprisonment after a single mistake may be a suitable punishment for humans, but is not one in the API space, as even a much better model is bound to cause a mistake at some point.
4. Another right question to ask is "how does effectiveness of AI at therapy in 2025 compare to the effectiveness of AI at therapy in 2023?" Where it's at right now does't matter, what matters is where it's going. If it continues at the current rate of improvement, when, if ever, will it surpass an average (or a particularly bad) licensed human therapist?
5. And if this happens and AI genuinely becomes better, are we sure that legislators and therapists have the right incentives to accept that reality? If we pass a law prohibiting AI therapy now, are we sure we have the mechanisms to get it repealed if AI ever gets good enough, considering points 1-3? If the extrapolated trajectory is promising enough (and I have not run the necessary research, I have no idea if it is or not), maybe it's better to let a few people suffer in the next few years due to bad advice, instead of having a lot of people suffer forever due to overzealous regulation?
What does this mean?
Not sure if people aren’t thinking that through or if they’re vastly overestimating the trustworthiness and transparency of your average professional human.
We used to worry about Bitcoin, now Google is funding nuclear plants.
on the other hand probabilistic/non-deterministic model, which can give 5 different advises if you ask 5 times.
So who do you trust? Until determinicity of LLM models gets improved and we can debug/fix them while keeping their deterministic behavior intact with new fixes, I would rely on human therapists.
I've never spoken to a therapist without paying $150 an hour up front. They were helpful, but they were never "in my life"--just a transaction--a worth while transaction, but still a transaction.
Do you have access to all the training data and the reinforcement learning they went through? All the system prompts?
I find it impossible for a company seeking profit to not build its AI to maximize what they want.
Interact with a model that's not tuned and you'll see the stark difference.
The matter of fact is that we have no idea what we're interacting with inside that role-play session.
You trust Anthropic that much?
Many dogs are produced by profit motive, but their owners can have interactions with the dog that are not about profit.
It would meet objective definition if you replaced 'capitalist' with 'socialist', which may have been what you meant, but that's merely an observation I make, not what you actually say.
The entire paragraph is quite contradictory, and lacks truth, and by extension it is entirely unclear what you mean, and it appears like you are confused when you use words and make statements that can't meet their definition.
You may want to clarify what you mean.
In order for it to be 'capitalist' true to its definition, you need to be able to achieve profit with it in purchasing power, but the outcomes of the entire business lifecycle resulting from this, taken as a whole, instead destroy that ability for everyone.
The companies involved didn't start on their merits seeking profit, they were funded by non-reserve debt issuance or money-printing which is the state picking winners and losers.
If they were capitalist they wouldn't have released model weights to the public. The only reason you would free a resource like that is if your goal was something not profit-driven (i.e. contagion towards chaos to justify control or succinctly totalism).