Hacker News Reader

Comment by Permit | Hacker News Reader

> "The incentive is to keep you online," Stanford University psychiatrist Nina Vasan told Futurism. The AI "is not thinking about what is best for you, what's best for your well-being or longevity... It's thinking 'right now, how do I keep this person as engaged as possible?'"

Is this actually true? Or is this just someone retelling what they’ve read about social media algorithms?

View on HN

I had a break from ChatGPT for a few months and got back onto it last week with some questions about game engines. I noticed that this time it's asking a lot of stuff when it looks like I'm coming to the end of my questions - like, "would you like me to go through with..." or "would you like me to help you with setting up..."

Previously it felt less this way but it was notable as it seemed to sense I was coming towards the end of my questions and wanted me to stick around.

I think you are referring to the 'ask follow-up questions' toggle in the settings but it's not available on all devices to turn it off.

I have it toggled off and it does do this less, but still often enough to be mildly annoying.

From an economic standpoint probably not.

The individual queries cost real money. They want you to like the service and pay for it, but there's not much in it for OpenAI for you to use it obsessively beyond training data.

They're basically simplifying and romanticizing how RHLF works.

https://openai.com/index/sycophancy-in-gpt-4o/

https://www.anthropic.com/news/towards-understanding-sycopha...

I find that more often than not the LLM will try to keep the conversation going instead of ending it.

It's not "thinking" in any sense of the word. As any LLM about budget date ideas in your city, for example, and watch them come up with the most cookie-cutter, boring, cringe filled content you've ever seen. Like blog spam, but condensed into a hyper friendly summary that optimizes for maximum plausibility and minimum offensiveness.

It's an extreme stretch to suggest that there is any thinking involved.

I doubt it's true yet but give it time.

Nah, it's just academic slop of the type every journalist has a crack-level addiction to. OpenAI's incentives are the exact opposite: users pay them a flat fee (or nothing) but OpenAI's cost scale per interaction. OpenAI make more money when people subscribe but don't talk to ChatGPT much, i.e. their incentives are the inverse of what Vasan is claiming here.

Ironically, the Stanford psychiatrist is hallucinating some statistically likely words whilst misinforming readers, perhaps in a way that will make them paranoid. It's turtles all the way down.

Reply on news.ycombinator.com