Hacker News Reader

Training on bot data can be bad when it's ungrounded and basically hallucinations on hallucinations.

Having LLMs help curate something grounded is generally reasonable. Functionally, it's somewhat similar to how some training is using generated subtitles of videos for training video/text pairs; it's very feasible to also go and clean those up, even though it is bot data.

Pixart-alpha provides an example of C: The bot labels can be dramatically better than the human labels.

Even though they used LLaVA, and LLaVA isn't all that good compared to gpt-4.

> Yup. This is my gut to where LLMs will really explode

Yes, indeed. This is one place where LLMs can make it look like a bomb went off.

We can do polynomial regression of data sets that looks equally plausible, but it's not real data.

> A: it's happening anyway

This is never a valid defense for doing more of something.