So, a small proportion of articles were detected as bot-written, and a large proportion of those failed validation.
What if in fact a large proportion of articles were bot-written, but only the unverifiable ones were bad enough to be detected?
What if in fact a large proportion of articles were bot-written, but only the unverifiable ones were bad enough to be detected?
But it looks like Pangram is a text classifying NN trained using a technique where they get a human to write a body of text on a subject, and then get various LLMs to write a body of text on the same subject, which strikes me as a good way to approach the problem. Not that I'm in anyway qualified to properly understand ML.
More details here: https://arxiv.org/pdf/2402.14873
Anecdotally speaking, I'm heavily involved in AI cleanup work on Wikipedia. At this point, I can read article text typical of AI, guess within 6-12 months when that edit was made based on differences in LLM output over the years, and usually be right. (e.g., "delve" was common through early 2024 until GPT-4o killed it; I've seen theories that it's not a GPT-4o thing but rather the result of people avoiding the word when it became a meme, but its frequency dropped off hard even in edits that clearly were not reviewed or edited at all.) Given that Wikipedia has accumulated billions of edits over 25+ years, that says a lot.