I know someone that has "resurrected" a dead relative through an llm, and I've seen the nonsense on forums about dating an "ai boyfriend"...
Some people can't help themselves and don't use these tools appropriately.
Some people can't help themselves and don't use these tools appropriately.
I've found Claude to be an excellent tool to facilitate introspective psychoanalysis. Unlike most human therapists I've worked with, Claude will call me on my shit and won't be talked into agreeing with my neurotic fantasies (if prompted correctly).
It's called unconscious intention, and here's a pretty interesting paper that'll bring you up to speed: https://irl.umsl.edu/cgi/viewcontent.cgi?article=1206&contex...
In your provided example, the user is obviously not trying to manipulate someone into questioning their sanity, nor power of reasoning. Quite the opposite. Lying to themselves (your example) for sure.
https://philpapers.org/rec/MCGAIG
https://www.psychologytoday.com/us/blog/emotional-sobriety/2...
It can drive people further and further into their own personal delusions or mental health problems.
You may think it's being critical of you, but it's not. It's ultimately interacting you on your terms, saying what you want to hear when you want to hear it. That's not how therapy works.
That's been my experience with human therapists. When I tell Claude to stop being sycophantic, it complies. When I tell a human to stop being sycophantic, they get defensive.
I agree that an ideal human therapists would be better than Claude, but most that I've worked with are far from ideal. Most are not very bright, easily manipulated, and quick to defensiveness when questioned. And Claude won't try to get me to take random meds with the only justification for the specific medication being 'got to start somewhere'.
To be fair these are probably the same people who would have been having these conversations with squirrels in the park otherwise.
But that isn't what I've seen done when people said they did that. Instead they just told ChatGPT a bit about the person and asked it to playact. The result was nothing like the person-- just the same pathetic ChatGPT persona, but in their confusion, grief, and vulnerability they thought it was a recreation of the deceased person.
A particularly shocking and public example is the Jim Acosta interview of the simulacra of a parkland shooting victim.
"but it's getting cheaper and cheaper to run inference" they say. To which I say:
ok bud sure thing, this isn't a technology governed by Moore. We'll see.
But that's not what I'm talking about, I'm talking specifically about people who've made a SoTA model their buddy, like the people who were sad when 4o disappeared. Users of character.ai. That sort of thing. It's going to get very, very expensive and provides very little value. People are struggling with rent. These services won't be able to survive, I hope, purely through causing psychosis in vulnerable people.
you haven't met the Whales (big spenders) in the loneliness-epidemic industry (e.g. OnlyFans and the like)
It's not like the "real women" of OnlyFans are consistently real, or women. And there's some percentage that are already AI-by-proxy. There's definitely opportunity for someone to just skip the middleman.