Hacker News Reader

Comment by bugglebeetle | Hacker News Reader

GPT-4 has been made worse in the ChatGPT UI as well since the May update. It makes many more strange errors and has trouble reasoning around complex problems and ambiguity. Prompts similar to stuff that worked fine for me last month now require multiple iterations of feedback. I’d switch to using the API, but I’d go over the equivalent $20 of usage pretty quickly.

View on HN

The models tend to degrade when trained to be safer.

A GPT-4 talk on youtube by personnel from Microsoft has documented this phenomenon with the 'Tikz Unicorn' evolution shown in the GPT-4 technical paper. The model gets qualitatively better with more training, and then degrades when trained to be safer (against racism sexism, etc), but it is not entirely clear why. These would seem very unrelated, especially when considering work done in LM editing (ROME/MEMIT) and the decent localization of knowledge seen there.

So, perhaps both the "I'm sorry I can't..." and 'strange errors' are not entirely orthogonal.

It seems pretty logical to me. Fine-tuning to make it more polite is giving it questions and punishing for giving an actual answer.

Probably not unlike people then. If you tell the truth you’ll be more often than not punished for it if you’re not very careful.

I find capitalism idiotic and broken, but I’m rarely allowed to say it, even if many people secretly agree with me, it might mean I’m a “communist” :)

To me it is clear why. Imagine someone told you "answer immediately, top of your head: what's the best seasoning?". You'd just blurt out whatever specific you associated with pleasing seasoning (and that would be a good answer). Now imagine someone said "answer immediately, off the top of your head, but without offending any culture, gender, without a cultural bias, and without being presumptuous of the listener's socio-economic status (and if you fail one of these, someone dies) what's the best seasoning?" Even without the way that is going to lead to all sorts of compromising and second guessing in the answer space, simply only a fraction now of your brain is left to associate about the question due to just holding all that other stuff in there.

I was just asking myself that yesterday...

Reasoning has degraded. To the point it was sometimes weirdly losing context and hallucinating...

Like its brain got fatigued or something...

Do we know if this is exclusive to ChatGPT? Is the API exempt from this issue?

Yes! I’ve noticed this too, it’s just slightly less sharp. It probably has to do with how much trouble they are having servicing all of the demand, so they have rolled out a scaled down version that requires less compute.

Reply on news.ycombinator.com