People are familiar enough with LLMs to understand the nuance in their limitations. Many of these only become obvious when you transition from using it as a toy to using it as a tool.

We a novel tool like this comes out people take wild guesses about how to use a tool, jumping well past its capabilities. When they see limits, they jump back to zero (or close to zero) and try again.

Once your familiar with a tool, you know how to use it near its limits. Naturally, you’re going to feel more pain when you cross the limit. It won’t feel like you’re far away from a known good.

I suspect this is essentially all that’s going on. P

We trained a large language model and had it deployed for multiple month last year (Cedille.ai, largest open source LLM for French at the time). We had multiple companies experimenting with it to use it in production. There were multiple occurrences where we got feedback about "the model having gotten significantly better". But the model in prod was never altered.

It is probably catastrophic forgetting.

I have not noticed GPT-4 getting any worse! I also don’t use Twitter. Perhaps there is a correlation?

Maybe the AI really is learning from it's human inputs, thus becoming less intelligent...

What's likely changed is your expectations.

any link to openai refuting the claim?