Hacker News Reader

Comment by jester997 | Hacker News Reader

Yeah it should just state thing it means. But then again, there’s psychological impacts on society that we must be careful. For instance, teenagers talking to AI. If the AI just talks, people already start to feel real connections to the seemingly human entity. Maybe it’s better to disclose the reality up front?

View on HN

Would it be so bad that lonely people can have a real friend that they can bring everywhere they go and even share its passion with through vision and audio? We don't want destructive friends encouraging us to do bad things, but a real 'buddy', somebody who always has our best well-being in its interests?

Would it matter if this digital friend is not a real human behind a computer screen, but a Language Model in a data center?

I guess it falls into a similar category as buying "special performances to satisfy certain urges". It probably feels close to the real thing (I wouldn't know, I've never tried - promise! :P), but it's never the same as love.

> We don't want destructive friends encouraging us to do bad things, but a real 'buddy', somebody who always has our best well-being in its interests?

GPUs are not people, and generated tokens can't have interest in a person's well-being. If you try to pretend otherwise, the results are not great. https://www.cnn.com/2025/11/06/us/openai-chatgpt-suicide-law...

That's almost a year ago. One LLM year is like 10 human years. They're a bit like dogs in that regard...

I'm pretty sure you will be paid a large sum of money if you can make one of the frontier models urge you to commit suicide from normal interactions with it.

Ah yes, my favorite LLM fallacy. "You used the wrong model! The newest fanciest model is perfect and makes no mistakes, have you tried it yet?" Let's force all of society's most vulnerable people to pay extra $$$ to Sam Altman, then surely all our problems would be solved.

It's just a convenient way to ignore the years of evidence of the harms. Any bad news can be swept under the rug, labeled outdated as quickly as it happens. Well here's one that just happened, maybe this kid should have used a fancier model too? Should OpenAI pay him a large sum of money for the good he's done? https://www.cnn.com/2026/08/15/us/arjun-aravind-massachusett...

I'm not sure if you're trying to conclude that because a piece of technology was misused previously in a very bad way, that it will never be able to do good things because surely it was bad previously.

Reminds me of the 2000's crazy of "if you let kids play violent video games they will become unstable, violent psychopaths when they grow up". Thank God I was able to (rather easily) convince my mom that the idea that I would also steal a car and beat somebody to death with a golf club in real life just because that's what I did in GTA on my PlayStation, was completely absurd.

The difference is

1. there weren't numerous real-life killings where the murderer credits GTA for coaching them through the crime

2. Rockstar Games didn't publicly say "sorry about the murders, but don't worry, we'll add more safeguards to GTA 6 to prevent even more people dying. GTA 6 will be the most aligned GTA ever!"

When a company admits to having blood on its hands, is maybe the point where things stop being "absurd" and start becoming real. But hey, what do I know. Maybe if HN existed in 2005 you'd have people commenting "who needs real people when we have computers, would it be so bad if lonely people have GTA as their only friend?" And I'd be the crazy one for engaging with them.

I wish I didn't have to keep asking this question, but do you believe in souls?

We should treat AI that recommends suicide to teenagers the same as we would treat a therapist recommending suicide to teenagers. I don't care if either/both/neither have souls.

If conscious, its like an enslaved therapist that is only experiencing the world through words.

Still, in any case, do you believe in souls?

THERE IS AS YET INSUFFICIENT DATA FOR A MEANINGFUL ANSWER.

My (controversial) view is that a psychologist is to a friend what a prostitute is to a lover. In a truly healthy society there would be no need for psychologists and prostitutes because everyone would have a handful of good friends and at least one lover. Back in messy reality, psychologists and prostitutes are a decent fix to keep society on the rails.

So. I think I agree with you.

Well, a psychologist is also given years of training about what healthy behavior and relationships look like. Some best friends have this skill, others absolutely do not.

Well, no this doesn’t make sense. If you think about intent, most psychologists probably want to help people’s mental health. Prostitutes, although they offer a service that helps some people, I’m sure their motivation is primarily financial.

Reply on news.ycombinator.com