Comment by sReinwald | Hacker News Reader

Comment by sReinwald | Hacker News Reader

Parent Full thread

You "skimmed through it" by your own admission, but somehow feel comfortable enough to armchair diagnose everyone involved as having "severe mental health issues". Seriously, are you not ashamed?

Your advocacy of "just focus on mental health" rings particularly hollow when you advocate that we should just ignore a technology that's actively destroying people's mental health, driving them to suicide. The irony is off the charts here. So, please, let's be real - you don't care about mental health. You're scared someone might put a warning sticker on your new toys.

Even if every single person had a pre-existing mental condition (they didn't, and you'd know this if you had read the article) your logic is sociopathic and genuinely disturbing. "Vulnerable people exist. Therefore, companies bear no responsibility when their products exploit those vulnerabilities and harm them. Yay, we don't need to think about it anymore." Cool take. I guess it's fine if technology kills people as long as they might have an undiagnosed mental health issue.

This argument is patently ridiculous in any context other than a Hacker News discussion full of technophiles and accelerationists. Moving fast and breaking things is fine when it comes to your database - not so much when you're dealing with people's lives.

We don't dismiss food safety regulations because some people have allergies. We don't ignore faulty airbags because some people are terrible or reckless drivers. But when a chatbot instructs someone to stop taking prescribed medications, increase ketamine use, and tells them they can fly if they jump off a building - that's just a "mental health issue" and not a flaw in the product? No guardrails, warning labels or safety mechanisms necessary?

The fact that this predominantly affects vulnerable populations makes regulation MORE necessary - not less.

Nothing says, "let's focus more on mental health" than claiming it's "silly" to suggest a technology that actively coaches vulnerable people through suicide attempts might be worthy of regulation.

Agreed. It’s depressing to see the top comment as so dismissive of the real human effects of technology.

If ChatGPT is helping accelerate spirals in those with previously unseen mental health issues the answer isn’t “well they’re mentally unwell anyway”, it’s “what could ChatGPT, in its unique position, be doing to help these people”. I have to imagine an LLM is capable of detecting suicidal ideation in the context of a conversation.

Brother like 90% of the freaks in these threads are heavily invested into the current LLM bubble so of course they have to hand wave anything negative.

The funny thing is that I am, arguably, one of those "freaks" heavily invested in LLMs - in the sense that I am a very heavy and fairly enthusiastic user, not in the Angel Investor sense. I spend an embarrassing amount monthly on subscriptions and API costs. I run uncensored models locally, and I'm responsible for maintaining our AI infrastructure at work. I use them for everything from coding to creative writing to research. I'm about as far from an LLM Luddite as you can get.

But here's the thing: I'm also technically literate enough to understand what I'm dealing with. I know these are effectively stochastic parrots, not oracles offering glimpses into the threads of fate, or the Matrix. I've also done years of therapy, so I know how to engage with anything related to mental health with respect, and a healthy dose of skepticism. I'm not saying I'm not vulnerable - anyone is vulnerable to manipulation. But I have the tools to recognize when an LLM is hallucinating, or reinforcing unhealthy patterns.

However, it seems like the people in this article - and most people that I have talked to, even those in technical roles - didn't have that context. They thought they were talking to something that "knew more than any human possibly could." And, I think what makes it even more dangerous, they talked to something that is trained to express itself with authority, and to be helpful to users. LLMs aren't necessarily trained for nuance, or to say "I don't know", or to push back on someone experiencing a psychosis. We are all just human, and most humans don't generally like it when LLMs refuse, offer vague answers, and they don't like it when LLMs disagree. That's what got us into the mess with GPT-4o turning into an absolute sycophant a few weeks ago. (https://openai.com/index/sycophancy-in-gpt-4o/)

But, being pro-LLM doesn't mean one has to oppose basic safety measures. I don't want my tools neutered or censored, but I also don't want them instructing vulnerable people to stop taking their meds or encouraging them to jump off buildings.

The fact that I can safely use these tools, doesn't mean everyone can. Just like the fact that my dad being able to safely handle a chain saw doesn't mean that I won't accidentally cut my arm off with one.

When someone thinks ChatGPT has special access to hidden truths, they're not using it wrong - they're using it exactly as it presents itself: with confident authority about everything.

You're right that an LLM could theoretically detect suicidal ideation - the article even mentions that ChatGPT briefly showed Torres a mental health warning before it "magically deleted." So the capability exists, but the implementation is clearly broken.

The more profound issue is that, the way I see it, LLMs are fundamentally multiplicative forces. They amplify whatever you bring to the conversation.

If you're an experienced programmer with solid fundamentals, an LLM becomes a force multiplier - handling boilerplate while you focus on architecture. But if you lack that foundation and vibe code away, you'll get code that looks right but is likely riddled with subtle or not so subtle bugs you can't even recognize.

And I suspect that the same principle applies to mental health. If you approach an LLM with curiosity but healthy skepticism, it can be a useful thinking partner. But if you approach it while vulnerable, seeking answers to existential questions, it doesn't provide guardrails - it amplifies. It mirrors your energy, validates your fears, and reinforces whatever narrative you're constructing.

The "helpfulness" training worsens this. These models are optimized to be agreeable, to match your vibe, to keep you engaged. When someone asks "Am I trapped in the Matrix?" a truly helpful response would be "That sounds like you're going through something difficult. Have you talked to someone about these feelings?" Instead, ChatGPT goes, "Yes! You're special! You're the chosen one! Here's how to unplug! Any tall buildings nearby, Neo?"

Do not confuse yourself, skimming does not mean I did not ingest the article but I did speed read through it as it was not that well written.

Why would I feel ashamed? I am not saying the crisis itself is silly but the article leads the reader through a biased take. But let’s also be real, there is an high probability every single one of these folks was having some sort of issue prior to the engagement with ChatGPT. Would love to see more research in the area and actual quantification of the issue but until that happens your argument might as well extend to all parts of life. Have a mandatory psych evaluation every year. Limitations to your access to the internet or other things, maybe even forced institutionalization if you don’t pass.

In your case, "speed reading" and "skimming" are obviously just two flavors of "I didn't actually read this properly but still feel qualified to diagnose everyone involved."

You'd love to see more research? You must've been reading so fast, you breezed right past the Stanford Study, the MIT Media Lab Research and the analysis by McCoy showing GPT-4o affirmed psychotic delusions 68% of the time.

Or do you mean it in the sense of "we have to study this for at least 20 years and let thousands suffer and die before we can determine whether chatbots affirming people's delusions or coaching them on how to kill themselves is a bad thing"?

Your concerns about the slippery slope from "maybe chatbots shouldn't coach people through suicide" to "forced psychiatric evaluations" or "institutionalization" is genuinely unhinged. And, tellingly, it betrays your view of mental healthcare as fundamentally punitive and authoritarian rather than supportive or therapeutic. Big advocate for mental health, you are.

You realize we already regulate countless things to protect vulnerable people from mental harm, right?

We put warning labels on cigarettes. We have age ratings on media. We don't let casinos exploit problem gamblers. We don't let bartenders serve alcohol to visibly intoxicated people. We don't let therapists tell suicidal patients to jump off buildings. None of these regulations are authoritarian overreach - they're basic safety measures that exist because in a healthy society, vulnerable people deserve protection, not exploitation.

But apparently when it's time to consider holding tech to similar standards, we're supposed to just shrug and say, "well, they were probably crazy anyway."

Your position essentially boils down to: "Vulnerable people exist, therefore companies should be free to exploit them." That's not a mental health advocacy position - it's tech industry bootlicking dressed up as concern trolling.

That's why you should feel ashamed.

You’re clearly passionate about this, and I don’t fault you for that. But I’d suggest dialing back the aggression, rhetoric like “bootlicking” and “you should feel ashamed” doesn’t exactly invite thoughtful dialogue. It just shuts it down.

Now, to the substance: I never argued against any safety measures or regulations. I specifically said I’d welcome more research and quantification of the issue. You’re citing some early studies, that’s good, but it’s also fair to question how representative or conclusive they are before leaping to sweeping policy implications.

My point wasn’t that chatbots helping people kill themselves is fine. It’s that if we’re going to treat them like a public health threat, we should hold all tools and technologies that interact with people to the same standard and yes, that includes thinking carefully about how we approach mental health more broadly, not selectively.

If a vulnerable person spirals after a chatbot interaction, that’s serious. But it’s also worth asking: what led them to be that vulnerable in the first place, and where were the other support systems? LLMs may exacerbate, but they aren’t the root cause.

You’re framing this as a binary,either you care and want regulation, or you’re enabling exploitation. That’s a false choice. I’m saying the issue is complex, the data is early, and we shouldn’t let moral panic drive policy faster than evidence supports.

Let’s push for more transparency, better safeguards, and yes, real research. But let’s also keep the conversation civil.

You may be correct about the language not necessarily inviting very thoughtful discussion. But I didn't exactly expect one, when I engaged with someone who merely "skimmed" the article and put out an incredibly nuanced, though-provoking and discussion-inviting take like "mentally ill people, am I right?" and "this article is silly".

I'm throwing back what you put out. You don't get to claim that you wanted a thoughtful discussion when you open the discussion the way you did.

You claim you have "never argued against safety measures," yet your original comment literally called the article "silly" for merely leading readers to perhaps come to the conclusion that regulation of these technologies might be a sensible step.

You keep asking for "more research" despite the article citing multiple studies. How many more studies would you like to see? What's the acceptable death count before we move on from "early data" to "maybe we should do something"? Or, more importantly, is there genuinely any number of studies that would actually convince you that regulation would be prudent? Because, please correct me if I'm wrong, you don't seem like the type who would welcome any sort of regulation, for any reason.

Your "what led them to be vulnerable" question is particularly revealing. Nobody claimed ChatGPT creates mental illness from scratch. The issue is that it's actively harmful to vulnerable people. Food doesn't create nut allergies, but we still put "may contain traces of nuts" on food labels. We don't need to ask "why do people develop nut allergies" to recognize that nuts are causing them to go into anaphylactic shock, and they should be informed if a product is potentially going to kill them.

You say we should "hold all tools and technologies to the same standard" - but we do. That's literally what regulation IS. You're arguing for the outcome while opposing the mechanism. We regulate pharmaceuticals, medical devices, therapy practices, and yes, even social media platforms when they cause documented harm. There is a reason why children under 13 are not allowed to sign up to social media in the US.

The "complex issue" framing is a deflection here, just as it is with climate change denial. It's an appeal to the status-quo. To not change something that is fundamentally broken because it would inconvenience you, or the business' bottom line. Guess we'll have to keep drilling for oil, keep burning fossil fuels and let ChatGPT teach people how to kill themselves, ah well. Complexity doesn't preclude basic safety measures, like a simple warning label.

You want "transparency and better safeguards"? Cool. Me too. How do you propose to make it happen? Self-regulation famously worked great for big tobacco, so I'm sure it'll work great in this case.

PS: I love the motte-and-bailey retreat from "severe mental health issues" to "some sort of issue" to a vague "vulnerable". PPS: How did you like the tone of this one? Better?

Reply on news.ycombinator.com