So, good for professionals who want to spend lots of money on AI to be more efficient at their jobs. And, bad for casuals who want to spend as little money as possible to use lots of datacenter time as their artificial buddy/therapist.
I use the GPT models (along with Claude and Gemini) a ton for my work. And from this perspective, I appreciate GPT-5. It does a good job.
But I also used GPT-4o extensively for first-person non-fiction/adventure creation. Over time, 4o had come to be quite good at this. The force upgrade to GPT-5 has, up to this point, been a massive reduction in quality for this use case.
GPT-5 just forgets or misunderstands things or mixes up details about characters that were provided a couple of messages prior, while 4o got these details right even when they hadn't been mentioned in dozens of messages.
I'm using it for fun, yes, but not as a buddy or therapist. Just as entertainment. I'm fine with paying more for this use if I need to. And I do - right now, I'm using `chatgpt-4o-latest` via LibreChat but it's a somewhat inferior experience to the ChatGPT web UI that has access to memory and previous chats.
Not the end of the world - but a little advance notice would have been nice so I'd have had some time to prepare and test alternatives.
Carry it forward into your next experience with OpenAI.
Just a few days ago another person on that subreddit was explaining how they used ChatGPT to talk to a simulated version of their dad, who recently passed away. At the same time there are reports that may indicate LLMs triggering actual psychosis to some users (https://kclpure.kcl.ac.uk/portal/en/publications/delusions-b...).
Given the loneliness epidemic there are obvious commercial reasons to make LLMs feel like your best pal, which may result in these vulnerable individuals getting more isolated and very addicted to a tech product.
I think that is going to be an issue regardless of the model. It will just take time for that person to reset to the new model.
For me the whole thing feels like a culture shock. It was rapid change in tone that came off as being rude.
But if you had that type of conversations from the start it would have been a non-issue.
It is little more than the Rat Park Experiment, only in this American version, the researchers think giving more efficient and various ways of delivering morphine water is how you make a rat park.
Please tell me what comes next then?
Outside of work I sometimes user LLMs to create what amounts to infinitely variable Choose Your Own Adventure books just for entertainment, and I don't think that's a problem.
And I'm just kind of interested _how_ other people are doing all of this interactive fiction stuff.
I've tried taking my vague story ideas, throwing them at an AI, and getting half a chapter out to see how it tracks.
Unfortunately, few if any models can write prose as good as a skilled human author, so I'm still waiting to see if a future model can output customised stories on demand that I'd actually enjoy.
But also, one cannot speak for everybody, if it's useful for someone on that context, why's that an issue?
Marty: Well, that's a relief.
https://www.nytimes.com/2025/03/18/magazine/airline-pilot-me...
https://en.m.wikipedia.org/wiki/Germanwings_Flight_9525
"The crash was deliberately caused by the first officer, Andreas Lubitz, who had previously been treated for suicidal tendencies and declared unfit to work by his doctor. Lubitz kept this information from his employer and instead reported for duty. "
The scary part: It is very easy for LLMs to pick up someone's satisfaction context and feed it back to them. That can distort the original satisfaction context, and it may provide improper satisfaction (if a human did this, it might be called "joining a cult" or "emotional abuse" or "co-dependence").
You may also hear this expressed as "wire-heading"
Does the severity or excess matter? Is "a little" OK?
This also reminds me of one of Michael Crichton's earliest works (and a fantastic one IMHO), The Terminal Man
The conversational capabilities of these models directly engages people's relational wiring and easily fools many people into believing:
(a) the thing on the other end of the chat is thinking/reasoning and is personally invested in the process (not merely autoregressive stochastic content generation / vector path following)
(b) its opinions, thoughts, recommendations, and relational signals are the result of that reasoning, some level of personal investment, and a resulting mental state it has with regard to me, and thus
(c) what it says is personally meaningful on a far higher level than the output of other types of compute (search engines, constraint solving, etc.)
I'm sure any of us can mentally enumerate a lot of the resulting negative effects. Like social media, there's a temptation to replace important relational parts of life with engaging an LLM, as it always responds immediately with something that feels at least somewhat meaningful.
But in my opinion the worst effect is that there's a temptation to turn to LLMs first when life trouble comes, instead of to family/friends/God/etc. I don't mean for help understanding a cancer diagnosis (no problem with that), but for support, understanding, reassurance, personal advice, and hope. In the very worst cases, people have been treating an LLM as a spiritual entity -- not unlike the ancient Oracle of Delphi -- and getting sucked deeply into some kind of spiritual engagement with it, and causing destruction to their real relationships as a result.
A parallel problem is that just like people who know they're taking a placebo pill, even people who are aware of the completely impersonal underpinnings of LLMs can adopt a functional belief in some of the above (a)-(c), even if they really know better. That's the power of verbal conversation, and in my opinion, LLM vendors ought to respect that power far more than they have.
Eh, ChatGPT is inherently more trustworthy than average if simply because it will not leave, will not judge, it will not tire of you, has no ulterior motive, and if asked to check its work, has no ego.
Does it care about you more than most people? Yes, by simply being not interested in hurting you, not needing anything from you, and being willing to not go away.
One of the important challenges of existence, IMHO, is the struggle to authentically connect to people... and to recover from rejection (from other peoples' rulers, which eventually shows you how to build your own ruler for yourself, since you are immeasurable!) Which LLM's can now undermine, apparently.
Similar to how gaming (which I happen to enjoy, btw... at a distance) hijacks your need for achievement/accomplishment.
But also similar to gaming which can work alongside actual real-life achievement, it can work OK as an adjunct/enhancement to existing sources of human authenticity.
LLMs cannot conform to that rule because they cannot distinguish between good advice and enabling bad behavior.
The real problem is that we can’t tell when or if we’ve reached that point. The risk of a malpractice suit influences how human doctors act. You can’t sue an LLM. It has no fear of losing its license.
* Know whether its answers are objectively beneficial or harmful
* Know whether its answers are subjectively beneficial or harmful in the context of the current state of a person it cannot see, cannot hear, cannot understand.
* Know whether the user's questions, over time, trend in the right direction for that person.
That seems awfully optimistic, unless I'm misunderstanding the point, which is entirely possible.
I understand this as a precautionary approach that's fundamentally prioritizing the mitigation of bad outcomes and a valuable judgment to that end. But I also think the same statement can be viewed as the latest claim in the traditional debate of "computers can't do X." The credibility of those declarations is under more fire now than ever before.
Regardless of whether you agree that it's perfect or that it can be in full alignment with human values as a matter of principle, at a bare minimum it can and does train to avoid various forms of harmful discourse, and obviously it has an impact judging from the voluminous reports and claims of noticeably different impact on user experience that models have depending on whether they do or don't have guardrails.
So I don't mind it as a precautionary principle, but as an assessment of what computers are in principle capable of doing it might be selling them short.
Is that just your gut feel? Because there has been some preliminary research that suggest it's, at the very least, an open question:
https://neurosciencenews.com/ai-chatgpt-psychotherapy-28415/
The second is "how 2 use AI 4 therapy" which, there's at least one paper for every field like that.
The last found that they were measurably worse at therapy than humans.
So, yeah, I'm comfortable agreeing that all LLMs are bad therapists, and bad friends too.
which is definitely worse than not going to a therapist
Here's a gut-check anyone can do, assuming you use a customized ChatGPT4o and have lots of conversations it can draw on: Ask it to roast you, and not to hold back.
If you wince, it "knows you" quite well, IMHO.
I keep the AI at arms' length; I know it doesn't think per se, but I enjoy the illusion.
> All LLMS are bad friends and therapists
That said it would not surprise me that LLMs in some cases are better than having nothing at all.
I do not think we need to imagine this one with stories of people finding spirituality in llms or thinking they have awakened sentience while chatting to the llms are enough, at least for me.
Sycophantic agreement (which I would argue is still palpably and excessively present) undermines its credibility as a source of independent judgment. But at a minimum it's capable of being a sounding board echoing your sentiments back to you with a degree of conceptual understanding that should not be lightly dismissed.
Here's my take further down the thread: https://news.ycombinator.com/item?id=44840311
An LLM is a language model and the gestalt of human experience is not just language.
Not everyone needs the deepest, most intelligent therapist in order to improve their situation. A lot of therapy turns out to be about what you say yourself, not what a therapist says to you. It's the very act of engaging thoughtfully on your own problems that helps, not some magic that the therapist brings. So, if you could maintain a conversation with a tree, it would in many cases, be therapeutically helpful. The thing the LLM is doing, is facilitating your introspection more helpfully than a typical inanimate object. This has been borne out by studies of people who have engaged in therapy sessions with an LLM interlocutor, and reported positive results.
That said, an LLM wouldn't be appropriate in every situation, or for every affliction. At least not with the current state of the art.
What a confusing sentence to parse
Anyone that remembers the reaction when Sydney from Microsoft or more recently Maya from Sesame losing their respective 'personality' can easily see how product managers are going to have to start paying attention to the emotional impact of changing or shutting down models.
Stories are being performed at us, and we're encouraged to imagine characters have a durable existence.
If you want an LLM to retain the same default personality, you pretty much have to use an open weights model. That's the only way to be sure it wouldn't be deprecated or updated without your knowledge.
Consider the implementation: There's document with "User: Open the pod bay doors, HAL" followed by an incomplete "HAL-9000: ", and the LLM is spun up to suggest what would "fit" to round out the document. Non-LLM code parses out HAL-9000's line and "performs" it at you across an internet connection.
Whatever answer you get, that "personality" is mostly from how the document(s) described HAL-9000 and similar characters, as opposed to a self-insert by the ego-less name-less algorithm that makes documents longer.
For example, keep the same model, but change the early document (prompt) from stuff like "AcmeBot is a kind and helpful machine" to "AcmeBot revels in human suffering."
Users will say "AcmeBot's personality changed!" and they'll be half-right and half-wrong in the same way.
The document or whatever you'd like to call it is only one part of the story.
I brought up prompts as a convenient way to demonstrate that a magic-trick is being performed, not because prompts are the only way for the magician to run into trouble with the illusion. It's sneaky, since it's a trick homo narrans play on ourselves all the time.
> The document or whatever you'd like to call it is only one part of the story.
Everybody knows that the weights matter. That's why we get stories where the sky is generally blue instead of magenta.
That's separate from the distinction between the mind (if any) of an LLM-author versus the mind (firmly fictional, even if possibly related) that we impute when seeing the output (narrated or acted) of a particular character.
It's Reddit, what were you expecting?
I think LLMs are amazing technology but we’re in for really weird times as people become attached to these things.
I’m less worried about the specific complaints about model deprecation, which can be ‘solved’ for those people by not deprecating the models (obviously costs the AI firms). I’m more worried about AI-induced psychosis.
An analogy I saw recently that I liked: when a cat sees a laser pointer, it is a fun thing to chase. For dogs it is sometimes similar and sometimes it completely breaks the dog’s brain and the dog is never the same again. I feel like AI for us may be more like laser pointers for dogs, and some among us are just not prepared to handle these kinds of AI interactions in a healthy way.
And probably close to wrong if we are looking at the sheer scale of use.
There is a bit of reality denial among anti-AI people. I thought about why people don't adjust to this new reality. I know one of my friends was anti-AI and seems to continue to be because his reputation is a bit based on proving he is smart. Another because their job is at risk.
https://www.reddit.com/r/MyBoyfriendIsAI/
They are very upset by the gpt5 model
these things are going to end up in android bots in 10 years too
(honestly, I wouldn't mind a super smart, friendly bot in my old age that knew all my quirks but was always helpful... I just would not have a full-on relationship with said entity!)
Seeing human-like traits in pets or plants is a much trickier subject than seeing them in what is ultimate a machine developed entirely separately from the evolution of living organisms.
We simply don't know what its like to be a plant or a pet. We can't say they definitely have human-like traits, but we similarly can't rule it out. Some of the uncertainty is in the fact that we do share ancestors at some point, and our biology's aren't entirely distinct. The same isn't true when comparing humans and computer programs.
You can also more or less apply the same thing to rocks, too, since we're all made up of the same elements ultimately - and maybe even empty space with its virtual particles is somewhat conscious. It's just a bad argument, regardless of where you apply it, not a complex insight.
Also worth noting is that alongside the very human propensity to anthropomorphize, there's the equally human, but opposite tendency to deny animals those higher capacities we pride ourselves with. Basically a narcissistic impulse to set ourselves apart from our cousins we'd like to believe we've left completely behind. Witness the recurring surprise when we find yet another proof that things are not by far that cut-and-dry.
I have the utmost confidence that things are only going to get worse from here. The world is becoming more isolated and individualistic as time progresses.
Unlike those celebrities, you can have a conversation with it.
Which makes it the ultimate parasocial product - the other kind of Turing completeness.
https://www.reddit.com/r/ChatGPT/comments/1mkobei/openai_jus...
There's already indication that society is starting to pickup previously "less used" english words due to AI and use them frequently.
https://www.scientificamerican.com/article/chatgpt-is-changi...
You can immediately identify them based on writing style and the use of CAPITALIZATION mid sentence as a form of emphasis.
It reads more like angry grandpa chain mail with a "healthy" dose of dementia than what you would typically associate with terminally online micro cultures you see on reddit/tiktok/4chan.
Just because AI is different doesn't mean it's "sad and cringe". You sound like how people viewed online friendships in the 90's. It's OK. Real friends die or change and people have to cope with that. People imagine their dead friends are still somehow around (heaven, ghost, etc.) when they're really not. It's not all that different.
We have automated away the "influencer" and are left with just a mentally ill bank account to exploit.
It shouldn't be so much of an ask to at least give people language models to chat with.
If they are real, then what kind of help there could be for something like this? Perhaps, community? But sadly, we've basically all but destroyed those. Pills likely won't treat this, and I cannot imagine trying to convince someone to go to therapy for a worse and more expensive version of what ChatGPT already provides them.
It's truly frightening stuff.
Maybe AI... shouldn't be convenient to use for such purposes.
The chats were heartbreaking, from the logs you could really tell he was fully anthropomorphizing it and was visibly upset when I asked him about it.
All AI has to be is mildly but not overly sycophantic and as a supporter/cheerleader to someone, or who affirms your beliefs. Most people like that quality in a partner or friend. I actually want to recognize OAI courage in deprecating 4 because of it sycophancy. Generally I don't think getting people addicted to flattery or model personalities is good
Several times I've had people speak about interpersonal arguments and them having felt vindication when chatgpt takes their side, I cringe but it's not my place to tell them chatgpt is meant to be mostly agreeable.
Perhaps if somebody were to shut down your favourite online shooter without warning you'd be upset, angry and passionate about it.
Some people like myself fall into this same category, we know its a token generator under the hood, but the duality is it's also entertainment in the shape of something that acts like a close friend.
We can see the distinction, evidently some people don't.
This is no different to other hobbies some people may find odd or geeky - hobby horsing, ham radio, cosplay etc etc.
I’m not sure I understand. You think the negative impacts are a good sign?
Perhaps you should read this and reconsider your assumptions.
Not every gaming subculture is healthy one. Plenty are pretty toxic.
> This is no different to other hobbies some people may find odd or geeky
It is quite different, and you yourself explained why: some people can’t see the distinction between ChatGPT being a token generator or an intelligent friend. People aren’t talking about the latter being “odd or geeky” but being dangerous and harmful.
They may stop making new episodes of a favoured tv show, or writing new books, but the old ones will not suddenly disappear.
How can you shut down cosplay? I guess you could pass a law banning ham radio or owning a horse, but that isn’t sudden in democratic countries, it takes months if not years.
Are you saying you're asocial?
This probably why I am absolutely digging GPT-5 right now. It's a chatbot not a therapist, friend, nor a lover.
GPT 5 is the first model I've used that has consistently done as it is told and returned only the changes.
I'm hoping that within a year or two machines like that will have dropped further in price.
One of Claude's "categories" is literally "Life Advice."
I'm often using copilot or claude to help me flesh out content, emails, strategy papers, etc. All of which takes many prompts, back-and-forth, to get to a place where I'm satisfied with the result.
I also use it to develop software, where I am more appreciative of the "as near to pure completions mode" as I can be mot of the time.
I needed some help today and it's messages where shorter but also detailed without all the spare text that I usually don't even read.
“Tackled” is misleading. “Leveraged to grow a customer base and then exacerbated to more efficiently monetize the same customer base” would be more accurate.