Hacker News Reader

Today, LLMs learn from a codebase that has mostly insightful comments from well-meaning humans.

In the future, the training sets will contain more and more automatically generated stuff I believe will not be curated well, leading to a spiral of ever declining quality.

View on HN

I am not overly concerned by a future where AI spoils its own training sets.

Companies like openai will keep injecting human-generated feedback into the training set.

A lot of the value is already in the RLHF today. See openai technical report.

Same reasoning suggests to make a copy of Wikipedia before it’s too late :)

https://dumps.wikimedia.org/

Would be interesting to analyze it over the next years, maybe even anlyzing the past ones, with these AI detection tools.

Remember that dude that wrote the vast majority of Scots wikipedia while not being able to speak Scots? Yeah, just waiting for the first LLM to make headlines by hallucinating it's own version of human history and publishing it to wikipedia.

Reply on news.ycombinator.com