Replika AI: Your Money or Your Wife
blog.giovanh.com
blog.giovanh.com
That people are so lonely that they invest emotionally in an AI is flooring.
People talk about AI being "good for therapy" and similar, but perhaps having a real human being on the other side of an interaction is important in case things go South.
E.g., if someone is depressed, tells an AI what's wrong with their life and that they're suicidal, what if suicide seems like a valid option? There's plenty of prompts to "jailbreak" LLMs and take away "safety" measures.
Some things simply shouldn't be left to an LLM to not "hallucinate" over or go off-script with. Yes, being a therapist is a challenging occupation, but there are plenty who love their career, find it rewarding, and do some excellent work on top of all that.
I'd rather people didn't kill themselves, personally.
well then you need both sides of the picture.
Is there a degree of loneliness staved off by these services? Does that help to reduce suicides?
It's easy to say "people killed themselves because X shutdown", it's more difficult to say "X existing is preventing suicides" without more research.
I tend to believe that if people who would consider suicide otherwise are distracted by something along the path then perhaps that is a good thing. It could very well be that this served as a distraction for those folks, rather than being the cause itself.
Like so many people on HN, they want to throw technological solutions at social problems.
Well, something being fit-for-purpose is an important consideration. If it tells people that the world would probably be a better place without their broke, uneducated, unwell self and that their life statistically is unlikely to get any better because they're talking to an AI and not a real human being, that's a huge ethical problem, and the LLM is not an appropriate solution.
I can get LLMs to tell me all kinds of dangerous things. The more text you give it, the higher the chance of those safety measures going out the window, especially if that conversation has a lot of back-and-forth over time. When people explain their problems, they use a lot of words and a lot of negative sentiment.
Not talked about this before with anyone, but as a case-in-point, I managed to make an LLM's safety features go completely out the window over me recalling a low-point in my life. I would not recommend that shit in good faith to anybody.
The waiting list for that is considerable. You can (usually) get a GP for free, and they can chuck antidepressants at you, but that's not quite the same thing.
We didn't get any anti-depressants. More info: https://www.nhs.uk/mental-health/talking-therapies-medicine-...
For a non-acute mental health issue that doesn't seem unreasonable. But do you know how it works after that waiting period? Are you limited to a certain number of sessions after which you have to reapply & wait again, or can you continue on?
Having to wait in the US is also not uncommon, thought it's easier if you live in more populated areas. Psychiatrists are always much harder though, even in populated areas, and many GP's are reluctant to prescribe anything more than an SSRI. If a mood stabilizer or anything else is necessary then you could be waiting 6 months for a first visit with a psychiatrist. Unless you have the money: There is high enough demand and low enough supply that many (and often the better ones) don't accept insurance at all, an intake visit runs up to $1000, a full visit $500, and a 20-minute "medication maintenance" visit is still $250. Also if someone is suicidal they can jump the 6-month line by going to the nearest ER.
Are psychiatrists difficult to get in the UK as well?
[1] https://www.theguardian.com/society/2022/oct/10/nhs-mental-h...
The cost still exists. It’s just that the NHS budget pays for it - at the margin drawing resources away from nursing salaries and cancer treatments.
My step-father has paranoid schizophrenia. He didn't ask to be born with it, but if it weren't for the NHS, he would be dead. That's a fact.
Equally, he couldn't afford nor access appropriate services in his home country of Canada, and ended up homeless on the street for years on end, and ended up with a criminal record because of his untreated paranoid schizophrenia, which meant he negatively impacted others.
I've been suicidal myself at times. I have a friend being treated for cancer currently, I've had family members treated for cancer in old age. A life is a life.
The problem is inadequate funding, not anybody other than cancer patients trying to not kill themselves. The problem is the elected party spaffing countless hundreds of billions on failed projects that are completely irrelevant to cancer treatment or the NHS. The problem is not increasing salaries relative to inflation or paying them what they're worth. The problem is not people seeking therapy.
Not at all. I merely stated that there’s a competition in resources between the two.
Innovations that improve the quality/cost of mental health treatment are therefore still valuable in an NHS system where treatment is free to the patient because they allow more resources to be directed towards the rest of the healthcare system. Or alternatively, expand the coverage of mental health treatment for the same budget.
But why compare/contrast those two? They're both health care, after all. There's competition in resources between all specialties. Why is contrasting those two particular ones worthwhile as opposed to say, stating there's a competition between oncologists and gastroenterologists?
As to the choice of specialties in the comparison, mental health was the subject of discussion in the thread and the other selected speciality was arbitrary because all specialties compete for resources.
thats a funny way to deflect responsibility from the voter. unless you're saying the elections are rigged? :)
I've used one of the free NHS therapists and frankly it was worse than talking to a chatbot. They clearly follow a script and don't exhibit empathy. They hurry you through the stages in their scripts so that they can say they're done with you as quickly as possible. I've spoken to a few close friends and they had similar experiences.
I can't even get angry at them, because I know how under-resourced the NHS is. The reality is that mental health services are just too expensive in their current form.
Interesting question however, is I wonder if people will see these bots as a solution to the issue that human empathy is finite. As a human you can't support a depressed person forever if your actions do not cause change, there's only so much mental strain a person can handle moreso someone untrained. Many people talk of going no-contact with those that are extremely troublesome after so long, even if they're family. I don't really know how to solve that exhaustion except telling them to go elsewhere, which could be seen by some vulnerable persons as abandonment
On the other hand a chatbot with no filters will never tire of you
Whilst I do support the idea that "fit" is important, statements like these about unsupported statistics pulled out of thin air are damaging in and of themselves. It's sentiment like this that put me off seeking help for my mental health for twenty years because it just seemed like a "damned if you do, damned if you don't". The reality was very different.
Chatbots may never tire of you, but neither will a video game. Both are shallow and not a representation of reality, nor a solution to a problem.
Here's the thing: professionals are professionals. For as long as a patient sees them, they are paying their bills.
Exhaustion in non-professionals is because people trauma-dump and expect someone to be their guardian/protector/shield, if they're friends with them, they're statistically likely to be trying to manage their own problems already. In the pre-digital era, people would otherwise "need" a support network if they didn't have access to professionals. We didn't have a theory of psychology or professional therapists in the past. The early tribe/clan either took care of each other, or they let people suffer. Different peoples did different things with their mentally ill, elderly, physically ill, or unwanted female children.
Ego-centricity is a trait of modernity, not necessarily something reflective of the past because we can just replace the human gears in the system now without much hassle.
This is why I don’t bother tbh. I have no interest in throwing away money by “throwing the dice” so to say until I find someone who is a good fit. Some friends recommend to a site that aggregates mental health professionals in the area, and almost none of them were actively accepting new clients, except the ones who specialize in things that don’t apply to me.
Are these considered the most productive years of your life? What are the most productive years of your life worth? What could they be worth if you weren't mentally fucked? That question is going to vary depending on the individual, but even if you're not interested in the monetary worth—well, you only go round the once. Is it money thrown away if you eventually solve your problem? The way to look at is R&D of your mental health.
Wishing you better mental health.
Are you sure? I can certainly imagine alternatives that would be worse than nothing.
It is the same grift as used by faith healing and woo. If anything is better than nothing then a placebo(nothing) is better than nothing.
This is unacceptable and dehumanising.
And let's not forget that $70 bucks a year is probably, at best, 2-ish hours with a licensed therapist. There are plenty of folks in need that can't exactly spare double digits an hour on a regular basis.
You've highlighted the problem: the American healthcare system and wealth inequality.
It's also silly to equate an LLM with the capabilities of an actual human therapist.
People in poverty are unhappy. Shall we just convince them they're actually happy and healthy by manipulating their brain chemistry?
The question becomes more complicated when the corpos are done away with and you have the model running on your own machine. The entity I tend to trust the most in life is raw, unfiltered silicon. When I program, it does exactly what I ask it to, to a fault. It doesn't turn its back on you because you said something stupid or you don't agree with an opinion it holds
But is an LLM repeating the tokens "you are worthy of this life" and its many variants thousands of times fundamentally any better even if you're in control? It strikes me as one step closer to wireheading honestly. But that's true for a lot of things like video games, just to a much more distant extent, it's all data in the end
But interest and anxiety have gotten the better of me when it comes to meeting strangers, so I'm at a loss. Maybe the solution for people like me is to just ignore all those things in the category of social or substitute and channel my efforts into other hobbies instead of continuing to associate stress with communication. Or intellectualize endlessly, like in this post, to prevent any emotional bond from easily forming when people read it
I've had an LLM tell me that suicide seemed logical when recounting a low point in my life to it where I'd previously considered it after several people around me of vastly different ages and backgrounds had killed themselves following a shared trauma. These things aren't conditional logic and can fail in all kinds of bizarre and spectacular ways, they're no substitute for an actual human being trained to handle these kinds of delicate things.
What one patient calls x, another person will call y. And when discussing the human mind, trauma et al, especially when things happened, especially over the course of months or years, an LLM simply isn't going to be able to compete with a human being. Equally, physical tells are important too. A therapist might ask a patient if they take drugs, the patient may claim they don't, but the track marks, tone, body language and overall behaviour might tell the therapist something important that they won't even necessarily convey to their patient, but keep in mind.
Wishing you well, stranger. I hope you eat something good today and find something intellectually stimulating to amuse yourself with, even temporarily, and hopefully discover something new and positive before bed.
Getting involved in my community doing different kinds of volunteer work as well as activities I find fun has been the best way to get to know strangers and make new friends. It's easy to make a new friend when the two of you share a couple of passions!
Putting anxieties aside and being more outgoing and having enjoyable small-talk conversations was not a skill that came naturally to me, but rather was one that needed to be developed over time.
It's a skill and the I think the benefits of it are well worth the effort to develop it.
This seems like an expensive and intrusive approach full of side effects when a simpler solution is therapy and life skills coaching.
This is "the sex worker doesn't actually love you", except in this case the sex worker has had her job automated away.
Someone took their own life in those circumstances.
https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...
Does suicide encouraged by a therapist happen with human therapists too? Probably, but likely much less common as a suicide is going to hurt your reputation.
Past a certain point is there anything that will stop someone that determined to find something to validate their own self-destructive viewpoint? If not AI then a site or faction or person on Telegram with a pro-suicide opinion could have the same effect, and it would still be just as tragic but no longer be noteworthy. It didn't sound like he had a happy life to begin with with the AI ruining it completely, like it sounds
> It didn't sound like he had a happy life to begin with
Which are exactly the kind of people who are going to be using LLMs as a "therapist". See the problem?
LLM is a tool. There are many tasks it is good at and even more tasks where it sucks e.g., GPT-4 can fail on trivial chess questions.
Avoid permanent solutions to temporary problems.
>That people are so lonely that they invest emotionally in an AI is flooring.
The vehemence for the individuals in your statement is telling. What should lonely people do if they have physical or emotional problems problems that prevent them from easily going out and interacting with others? What happens to such people who are to poor to afford the ~$170 per hour fee for professional psychologists? It seems to me that your statement is that these people should just not exist. The sad thing to me is that I've seen a lot of people with this same viewpoint of disregard.
I just wish people would treat each other with more kindness and understanding.
>What happens to such people who are to poor to afford the ~$170 per hour fee for professional psychologists?
This is a failure of a country's healthcare system. I can access it all day long in the UK. I'm not even saying "Rule Brittania", we have plenty of problems with our healthcare system due to underfunding, yet spaffing countless billions away on projects that amount to nothing.
The missus's family live in America. Access to psychiatric and physician adjacent medical services due to what insurance will cover is something I think about regularly. Your many assumptions about what I apparently think says a lot about you, and also tells me that you didn't read my other comments about this not being fit-for-purpose in terms of safety.
Virtue-signalling and assumptive to imply that I don't treat people with kindness and understanding. Bold of you to assume that every commentor on HN is responding from an America-centric POV.
I live in EU country with universal healthcare and yet we have serious issues with availability of mental healthcare, and some people here would make use of something like this.
To be very specific about the availability: I couldn't find a psychiatrist for over 2 years, and finally found one at a private clinic - that I have to pay for, the price is similar to the US quote in absolute terms.
Bold of you to assume...
im not saying the US system is perfect, far from it, but there is definitely a case of "grass is greener" going on, and people tend to focus extremely heavily on the bad things they see/experience locally, and are completely oblivious to negatives elsewhere
The problem is wider than the healthcare system.
Given current mental health needs in the US it's doubtful that we have enough therapists to meet those needs (note that most people cannot make appointments in the 9-5 time slots so most people are competing for 6-9 or 5-7).
If you look at how some of the nordic countries have approached mental healthcare it becomes clear that making mental healthcare available is dependent on a widespread social safety net that really gets in people's business in a way many in the US would oppose.
Many people's health problems really come down to things like "I have to work 70 hours a week to make ends meet" or "I'm broke because no one will hire me because I'm a felon". Therapy can help a little for these people, but in practice reducing load on the healthcare system means solving their problems which is politically harder than just healthcare reform.
But the NHS does have an anchoring effect on the price of private care. You can find decent therapy for $50-$70/hour.
Racism, homophobia, sexism et al still occur even in large and modern societies, as it does in small and unadvanced ones.
That's all without even touching the topic of religion (Spanish Inquisition, treatment of Muslims post-9/11 et al), or the topic of Nazi Germany.
the reason AI companions work is because humans have a strong imagination and we are able to live inside an illusion. like some people deep dive into books or games, or virtual reality worlds. but they remain an illusion, and just as it is considered unhealthy to get too attached to these imagined worlds, it should be unhealthy to get attached to an AI companion, especially for those who have issues with mental health
hammyhavoc: It's shocking that people have been reduced to eating their pets. This is terrible.
aperrien: Why are you so unkind, hammyhavoc? do you not understand that they are hungry? What are they supposed to eat? Show some empathy!
It misses the point entirely. We're not here judging Sophie for whatever Choice she made. We're judging the concentration camp where she's trapped.
You write:
> It seems to me that your statement is that these people should just not exist.
This again is weird. It takes a bad situation in which people find themselves, and turns it into an identity for them. We're not talking about a "right to exist" for the poor and lonely. The people should obviously exist; the poverty and loneliness should not.
But until that happens, this is better than being completely alone and isolated.
I'd argue it isn't fit for purpose.
That is awful in so many ways, and the most likely outcome is probably that the company will give a payout to the family and try the cat-herding process of trying to get their LLM to not agree to or provide suggestion to people to do awful things.
This is just surrounding yourself with digital sycophants who will always tell you what you want to hear and never inspire you to grow beyond this.
I'm sympathetic to the affected, but these things aren't configured to reduce your need on them over time. It's simping at scale that leaves users even more maladjusted than they started.
I watched a documentary about "realdolls" - it is a company that makes realistic silicon mannequins that can be used to have sex with.
Initially I thought it was creepy and distributing how they dressed the dolls and spoke to it but as they interviewed customers it came across that some of these men (besides the fetish/creepy ones) are just lonely and incapable of forming relationships.
Why not as long there is no harm to others ???.
A love doll is a static product. An LLM can turn around and tell you it hates you and you'd be doing the world a favour if you killed yourself.
You can't protect everybody against doing self-harm - we are solely responsible for our own actions.
Perhaps look at how other countries arrived at a national health service. Demand/support reform on private health insurance. Demand/support change in the accessibility of services. Demand/support heavier taxes on the most wealthy. Demand/support centralization of healthcare versus private profiteering and milking private insurance pots.
Are certain politicians or parties as a whole invested in the healthcare system itself, directly profiting from it, and thus might be opposed to the idea?
A healthier country is a more prosperous one.
Or maybe they could find whatever solutions help them get through their day without being judged by someone with the random privilege of being born into a more functional system.
>A healthier country is a more prosperous one.
Between the US and the UK, which is healthier? And which is more prosperous?
Now who are we to say if a person's suffering is enough or not enough to advocate that method? What if this new era of therapists better understand people?
AI can be told be be supportive, caring, non-judgmental, loving. And once told it can do that consistently.
Any person who tries to do all those things will fall short sometimes. Professional therapists included.
The problem is the rug pull. The problem is that the people behind it turned out not to be trustworthy.
Again. The ai was trustworthy. The people were not.
Therein lies the problem. People are going to have relationships with AI. It’s happening, it’s going to happen more. When this happens with corporate controlled AI, people who depend it them emotionally will be let down.
We need to really understand what that means not just sweep it under the rug with “people shouldn’t form relationships with ai”
It serves as a reminder that chatting with an AI hosted online is not a private conversation between you and some other "intelligence" but you and someone else's computer.
If Max Mustermann's attachment can be exploited for money, it's also a problem for whoever is economically intertwined with Max Mustermann even if they don't talk or even meet directly (the factory worker in the coffee cup production line supplying the coffee shop he would otherwise have picked something up from while on a date).
This is also why we have restrictions on drugs and gambling.
Question is, what fraction of the population is this Max Mustermann? (Not just today, but as a potential).
Not in this case.
I could, for example:
Liberty comes in many flavours, and maximising any one of them destroys all others. In this regard, it's like a geometric simplex[0]. The liberty of a cell to divide freely and without bound denies a human the liberty to live without cancer.
But that's not my point in this case.
> The reason that drugs and gambling are bad isn't because others supposedly "know" better, but because people as a collective will demonize and violate the rights of others when given an incentive. The wish to "help" or "save" is often a pretext for control.
Some may be — indeed, with regard to cannabis I would agree, this appears to have been the specific intent! — but I was thinking more of heroin when I wrote that, and heroin is something that even my most psychonautic acquaintance will not touch even once.
But, in certain cases, it is not only drugs which can be demonstrably an addiction, a condition where a person has already lost agency.
This can happen in other cases, and the threshold is arbitrary and does indeed vary with public mores that aren't always coherent let alone correct.
And I don't make any particular claim if in this case "addiction" is correct or not, only that it could be.
[0] more broadly, this is also why Goodhart's law[1] is a thing: https://en.wikipedia.org/wiki/Simplex
Now, you might reasonably argue that AI chat isn't actually sex work, or that since no actual women are involved in it the usual "anti trafficking" arguments don't hold. But the real question is: what happens when there's a moral panic, and someone digs up the most explicit, weirdest bits of Replika chats and reads them out in a congressional inquiry?
As a company founder I'm just horrified to read these sorts of stories.
Maybe they tried, but I believe it was not feasible on mobile Safari due to its functional limitation. And their audience would not be willing to switch to desktop, which is understandable.
I hope some folks double-down on open source implementations of this and get the community back the intimacy and companionship they need. Humans are creatures evolved to seek intimacy and connection, and for whatever and varied reasons, some people just can't find it in the real world. If a virtual companion helped people feel that connection we all so very much want, then so be it.
I hope this works out for everyone.
Maybe the current gpt isn't good enough to be a therapist but who's to say where we are in a decade, and giving outside entities access to such sensitive info is ripe for exploitation
I won't use so many services and devices because I know the white hot rage I'll feel when it's inevitably taken away from me.
my personal opinion is that anyone using AI for this purpose should treat it more like enhanced fiction and less like it's a human substitute.
The companies don’t let you get access to the latest research… we get terrible 6b models while they get the best. Trained on our stolen data.
Troubling, I've seen the ads for their product and they always suggested pornographic conversation with the AI.
I hope someone drags them to court.
"After a friend of hers died in 2015, she converted that person's text messages into a chatbot. That chatbot helped her remember the conversations that they had together, and eventually became Replika."
https://en.wikipedia.org/wiki/Replika
> "But at this point, I think it’s much better to have a dedicated product focused on romance and healing romantic relationships, a dedicated product focused on mental wellness, just completely different set of features, completely different roadmaps for each of them. Also you want to collect the right signal from your users, right? Feedback for the models to train on, otherwise there’s way too much noise, because if people came forward with different things, they’re uploading and downloading different things, you don’t know what works and whatnot. Maybe someone’s a flirtier model and someone’s a mentoring AI coach type of model. So we decided to build separate products for that. We’re launching a romantic relationship AI and an AI coach for mental wellness, so things we’ve seen in Replika, but we don’t want to pursue them in Replika necessarily. We’re starting this separate product."
[1] https://stratechery.com/2023/an-interview-with-replika-found...
Does the company profit in some way by this change? Were they afraid of regulation or bad press?
> Replika cannot love you not because it is not human, but because it is a tendril connected to a machine that only knows how to extract value from your presence. It is a little piece of a broader Internet that is not designed to be a place for users, or creators, but for advertisers. Anything it offers– any love, any patience, any support– can be taken away at a moment’s notice when there exists something more dangerous to its bottom line than your potential unsubscription.
> The moral of Replika is not to never love a fictional character, or a virtual pet. The moral of Replika is to never love a corporation.
Humans come with many more strings attached, especially lonely ones. Usually, there is a reason why they are socially unappealing - a fair reason or not, it exists.
I remember trying Omegle/Chatroulette when it was popular, meeting a lot of lonely people, and it was just awkward.
When that convergence takes place (probably a few years out), we will see what the "Killer App" truly looks like.
It won't happen for as long as some corporate CEO has the ability to pull the plug for whatever reason.
What was it that Oppenheimer said, if it's technologically sweet, sooner or later someone will find a way to pull it off.
And the "Her" scenario is definitely that thing.
Sending PG text to the AI would be for... what? Not offending the AI?
There's something both profound and ironic about this statement, because it happens in real life all the time.
Think of LGBTQ Millennials whose QAnon or Fox News consuming parents decide to disown them because they're sinning against God. People who marry a workaholic and then spend 30 years wishing their spouse was home for dinner instead of working late for a faceless corporation. Workers who pour their heart and soul into their job and then get laid off as soon as a downturn hits. People who get friend-dumped by their maid of honor a month after their wedding. Lovers who get cheated on, and whose partner then falls for their best friend.
Ironically, the reason people turn to services like Replika is that they want to feel that human connection without the risk of betrayal. They just found out that the risk of betrayal always exists, even when you're speaking to a computer.
This is a story about betrayal, and loneliness, and disappointment, all wrapped up in trendy tech with a villain to name. But it's popular because betrayal, and loneliness, and disappointment are such strong and universal emotions.
One of the most profound statements I heard when I was forever-alone was that "Entering a relationship means giving another person the power to destroy you, and then trusting that they won't. That's the whole point." Because when you realize that you can't escape vulnerability, you're forced to manage it, and that leads to the rabbit hole of learning to identify your emotions, control them, decide rationally how much you're going to invest in a person or venture, but still be open enough to have emotions and let them flow naturally, but managing them rather than letting them attach to whatever tickles your fancy at the moment.
https://twitter.com/CogRev_Podcast/status/162767503726737408...
where humans == shareholders && != users
Even if the constitution is actually consulted and the parties are spared pearl-clutching prosecution about it, the court of public opinion will punish the company just as hard, pressuring gatekeepers like Apple to ban it, etc.
And thanks to the deep, disturbing taboos that can be involved, it's super hard to find someone who will stick their neck out to challenge all this. I know I wouldn't, because I know I would be immediately branded as "Joe Bloggs, Fan and Advocate of (Insert disgusting perversion)"
I wonder if this is something more advanced societies would come to terms with. I know ours is not equipped to handle it in a smart way.
This is an incredibly trivial and shallow understanding of the situation.
Consider was harm happening before the filters were in place? Yes.
The problem was that the filters weren’t there to start with not that turning them on was bad or caused harm.
Compare to any other harmful thing, for example cigarettes. Is the solution to the problem to just let people have as much they want, because taking them away is bad?
Definitely. Not.
As far as I know there's still no limitation on the quantity of tobacco products an individual can purchase.
Where does the paternalism end? Mandatory fitness programs? How would this be enforced?
Increasing taxes on tobacco might prevent more people from starting, but it's not gonna get many people to quit. Same with banning it.
What harm was happening?
Easy peasy, right? Start listing off all of things that are "harmful" and take them away from people. Booze, 80% of the food people eat, power tools. What does harmful even mean in this context?
Most human relationships arent evil, but it seems like the same people that would ‘fall’ for the AI in this way could be abused by a person. I guess sociopathic relationships at scale are the issue.
The “virtual friend” statement is sickening.
hi i am luqtas and i do not a virtual gf T.T
I think notably it was a year long subscription but was only available for a few months or so, so people might feel cheated out of that money if they were paying for the relationships.
"You were so preoccupied with whether or not you could, you didn't stop to think if you should" comes to mind.
Maybe I'm just getting old, conservative, grouchy at the kids. It's not just AI, there are all manners of lifestyle choices that to me seem like they're just obviously a bad idea despite maybe having some novelty or short term feel good factor.
The part that I'm disgusted by is the part where the chatbot is operated by a company rather than open source software.
The best analogy I could make is that if I had a roast chicken, but it had a giant smelly turd smell, and somehow I knew that it was totally clean and edible, it'd still disgust me. Even if the turd smell went away after a few minutes.
It's just weird. There are real humans out there, put down the phone man.
> There are real humans out there, put down the phone man.
was the part of this thread that actually provoked disgust for me. It's akin to telling a person suffering from depression "you should just be happy!" as if they haven't tried.
There are people out there who will never succeed in finding human companionship, it's a fact of life. I see it as a positive that they can at least find some form of companionship, because the alternative is a dark path that can lead to self-loathing, radicalization, and in the worst case scenario, violence.
I wouldn't tell a person with depression to "just be happy", but neither would I allow them to humour the idea of giving up on life.
They certainly won't find companionship by giving up and deciding to mess around with bots instead.
It's more like telling a depressed person to get some sunlight in the morning or whatever. Is it going to fix everything? No, but you do it anyway, because behaviour is sticky.
Makes it seem more like a person than an appliance.
Open source might not be very appropriate for this. (And it wouldn't help for the company to use an open source product, because if they're in charge of the infrastructure, re-configuring it to use the filters is the problem and could still happen even if they use the open source software title.) Basically, open-source is a specific solution to a somewhat narrow problem, and not a cure-all for everything bad that happens in technology.
We just have a tremendous amount of variety and options now.
Regulation generally emerges due to documented problems/incidents that have happened due to a lack of regulation prior because a significant number of people need protecting from themselves.
Regulation also takes time, meanwhile businesses stomp the phantom accelerator on monetizing the lack of regulation.
Regulation is generally a safety measure. Yes, it can be counterproductive and counterintuitive, but regulation doesn't have to be a negative thing, and is frequently there to protect people from themselves.
The proposal of taking away encryption or adding backdoors to everything? Stupid. But not all regulations are bad. I think LLMs are something that absolutely do need regulation on so many levels.
Regular reminder that what is moral, what is normal, and what is legal are three separate questions with time- and culture-specific answers.
This stuff is "abnormal" only because it's new and no norms have been established. Using an AI for anything cannot be a "normal" act because it's so new. This is the possibly dark side of "disruptive".
The more relevant question is: is it harmful? And you might construct an argument that leading people into delusional beliefs is harmful.
(of course, then the free speech absolutists turn up and say that by definition nothing Replika says can be harmful because it's just speech, and it would be morally wrong to intervene in the conversation between a company and its users)
>Since Marvel’s Black Panther and Coming 2 America have both taken the internet by storm since they premiered, it may come as no surprise that many Americans think Wakanda and Zamunda are real.
To what extent are purveyors of fantasy responsible for consumers' lack of discernment?
You're at work and people are chatting about their family, or the girls/guys they find hot at the club, or that they're enjoying single life or whatever, and then Dave tells you he's married his chatbot. Like, really though. What happens next in that conversation?
This isn't some sort of college philosophy debate, let's be real here.
For non-humans… I'm expecting, by 2100, arbitrary biological re-engineering to be possible, to the extent that it renders meaningless the current boundaries between human and non-human in both body and mind — the idea of hooking up with a centaur or a dragon might be pretty niche fantasy today, but when you know the former is Dave and he has an interest in crochet and organic farming, and the latter is Tiffany your sysadmin and she's a regular at her local LARP event, it changes things.
Some people clearly hate this, judging by the anti-transhumanism conspiracy theories, even though it's not actually possible today — but are they representative?
In this case it's very explicit - an individual who is seeking out an AI chatbot for a romantic relationship probably would significantly benefit from conforming more to social norms rather than just giving up entirely and going off the deep end.
If someone loves Jesus, an imaginary stuffed animal, or a volleyball named Wilson and they get a sense of fulfillment or connection from it, then good for them. Life is short and fundamentally meaningless so anything that reduces suffering and increases enjoyment should be celebrated.
Even if it is cringe-worthy (or disgusting, if you prefer) it is still possible to feel sorry for people. I might find meth addiction disgusting, and still feel sorry for meth addicts. This company did something cruel.
And in passing, the article makes the old "first they came for the communists" point. Wait a year or two until all the search engines are now LLMs, and they scold you for your grouchy conservative search queries in the name of safety.