Hacker News Reader

Ok, yes, but that's not what model collapse is. I was expecting an interesting finding on LLM training stalling due to AI pollution of the internet (the training data). Instead this a rant about the pollution of the internet. All good points and it will lead to model collapse, but this is a clickbait headline.

View on HN

But, the internet is already polluted and being constantly polluted by LLMs. It feels like LLMs are the PFAS of the internet. So, I'm not sure what your argument is, but I would welcome clarification. Maybe I am misunderstanding your definition of model collapse. The article references this definition of model collapse[0]. I'm curious when you say "that's not what model collapse is" -- what do you mean by model collapse that is different from the definition of the article and reference?

[0] https://www.nature.com/articles/s41586-024-07566-y

My point is that their rant is not about that definition, even if they reference it. Again, they are not wrong in their concerns at all! It's just that having a PFAS'd internet is not a "sign of model collapse". A sign of model collapse would be new LLMs having degraded performance.

> A sign of model collapse would be new LLMs having degraded performance.

That's a good point. Stagnant performance would be a different thing -- more indicative of the technology itself.

Reply on news.ycombinator.com