Hacker News Reader

That one was (genuinely) a bug. OpenAI rolled it back. https://openai.com/index/expanding-on-sycophancy/

(But yeah, relying on systems that can have bugs like that for your mental health is terrifying.)

Yeah I think there is plenty of room for good discussion here, but using that quote without context is misleading. And the faulty model was pulled after only a few days of being out, iirc. It definitely does speak to the necessity of nuance when analysing AI in these contexts; results for one model might not necessarily hold for another, and even system prompts could change results.

you cannot really roll back a bug in a black box system you don't understand

Exactly. More like changing the state of the system to reduce the observed behaviour while introducing other (unknown) behaviours

I don't think that's quite true; even a system you don't understand has observable behaviour. And you can roll back to a certain state that doesn't exhibit the undesirable observable behaviour. If anything, most things in life operate this way.

Reply on news.ycombinator.com