AI Loses Its Mind After Being Trained on AI-Generated Data

You probably have to clarify that it only applies to the direct transformer training with fixed weights for samples, and not RL variants or variable sample weights.

Same for the 70's and 80's music cassette bootlegging scene. Some of the tapes were quite unlistenable because they were so degraded from being a copy of a copy of a copy.

The irony is that AI is great at cleaning them up!

If the headline explicitly anthropomorphizes a language model masquerading as AI, I doubt you're going to get anything reasonable by clicking through it.

LLM Generated content?

Best to just hear it straight from the horse's mouth.

https://arxiv.org/abs/2307.01850

Its potentially an even worse problem than overfitting because of the error accumulation and undermining of the fitness functions. Its similar to asking students to create test questions to evaluate themself. As the proportion of self output questions grows, the context eventually drifts to be meaningless. In other words it leads to test "questions" like "The answer is A".

One measurement of this would be to watch the growth of probability of new models to say similar things to older model filters like "As a large language model ..."

A definition of "mind" would help settle this.

Bah. I often say things have “lost their mind”.

My car stereo loses its mind once a month or so and I have to reboot it. (SYNC, it sucks)

One time my garage door kept refusing to close, intermittently and at a random spot in the cycle. “Stupid thing is going crazy”. (Mental metaphor still there)

Often times I’ll notice that my golf balls “have a mind of their own”

This is not a weird use of the idiom. I think it’s a fine headline.

I think the whole site is LLM-generated. There are articles where the headline is not even directly related to the content, and the content is... well, the problem is that it is nearly content-free.

For example, their super guide to prompt engineering excellence: start by getting a bachelor's degree in the field you want to prompt about...

Those that believe the I in AI actually exists might.

It's more boring to say LLMs trained on output from other LLMs fail to build viable statistical models of language.