This is a problem, because you can easily get stuck in a self-reinforcing loop. You feel strengthened in your convictions that you're good at ferreting out LLM-speak because you've found so much of it. And you find so much of it because you feel confident you're good at it. Nobody ever corrects you when you're wrong.
Combine that with general overconfidence and you get threads where every other post with correct grammar gets "called out" as AI generated. It's pretty boring.
There's a similar effect with contentious subject. You get reams and reams of posts calling the other side out for being part of a Russian/Israeli/Iranian/Chinese troll network. There's no independent falsification or verification for that, so people just get strengthened in their existing beliefs.
I honestly don’t know what sites like this will do when that happens and the only way of detecting LLMs is that they are subtly wrong or post too much, we’d be overrun with them.
Not sure if we should be hopefully or fearful that they will improve to be undetectable but I suspect they will.
And yet people seem to still be terrible at that. Someone uses an em-dash and there's always a moron calling it out as AI.
> I honestly don’t know what sites like this will do when that happens and the only way of detecting LLMs is that they are subtly wrong or post too much, we’d be overrun with them.
My personal take is that it doesn't really matter. Most posts are already knee-jerk reactions with little value. Speaking just to be talking. If LLMs make stupid posts, it'll be basically the same as now: scroll a bit more. And if they chance upon saying something interesting then that's a net gain.
Personally, I think it will matter deeply if sites like this are overrun by bots. If you believe your description, why are you here?
Similar to how you can watch one fantastic western/vampire/zombie/disaster/superhero movie and love it, but once Hollywood has decided that this specific style is what brings in the money, they flood the zone with westerns, or superhero movies or whatever, and then the tropes become obvious and you can't stand watching another one.
If (insert your favorite blogger) had secret access to ChatGPT and was the only person in the world with access to it, you would just assume that it's their writing style now, and be ok with it as long as you liked the content.
Overly focussed on style over content
Melodrama even when discussing the mundane
Attention grabbing tricks like binary opposites overused constantly
Overuse of adjectives and adverbs in particularly inappropriate places.
Lack of coherence if you’re generating large bits of text
General dull tone and lack of actual content in spite of the tricks above
Re your assertion at the end - sure if I didn’t know I’d think it was a particularly stupid, melodramatic human who didn’t ever get to the point and probably avoid their writing at all costs.
There's precious little training material left that isn't generated by LLMs themselves.
Consider this to be model collapse (i.e. we might be at the best SOTA possible with the approach we use today - any further training is going to degrade it).
Percentage-wise this is quite exaggerated.
> Consider this to be model collapse (i.e. we might be at the best SOTA possible with the approach we use today - any further training is going to degrade it).
You consider this above factor to lead to model collapse? You’ve only mentioned one factor here; this isn’t enough. I’m aware of the GIGO factor, yes. Still there are at least ~5 other key factors needed to make a halfway decent scaling prediction.
It is worth mentioning one outside view here: any one human technology tends to advance as long as there are incentives and/or enthusiasts that push it. I don’t usually bet against motivated humans eventually getting somewhere, provided they aren’t trying to exceed the actual laws of physics. There are bets I find interesting: future scenarios, rates of change, technological interactions, and new discoveries.
Here are two predictions I have high uncertainty about. First, the transformer as an architectural construct will NOT be tossed out within the next five years because something better at the same level is found. Second, SoTA AI performance advances probably due to better fine-tuning training methods, hybrid architectures, and agent workflows.
> Percentage-wise this is quite exaggerated.
How exaggerated?
a) The percentage is not static, but continuously increasing.
b) Even if it were static, you only need a few generations for even a small percentage to matter.
> You consider this above factor to lead to model collapse? You’ve only mentioned one factor here; this isn’t enough. I’m aware of the GIGO factor, yes. Still there are at least ~5 other key factors needed to make a halfway decent scaling prediction.
What are those other factors, and why isn't GIGO sufficient for model collapse?
Yes. People keep saying, in response to points like this, "oh but you/I can tell pretty easily." But it's not the detection, it's the verification! (see what I did there)
Where I'd push back is the idea that the problem is the boring "call out" discourse that follows each accusation. The problem of verifying human provenance is fundamental to the discussion of trust and argumentation, but the simple "the zone is flooded" problem is also an ecological one. There's terrible air/water/soil quality in the metro area I live in; people have to live with it w/o regard to how invested they are in changing it.
I cannot know what this says about my false negative rate, but at the very least I am confident in my false positive rate.
That's how a trope starts. When a minority of writers are using a particular pattern, it's personalized style. When a majority of writers in a genre adopt the same personalized style, it's a trope.
We find AI tropes especially annoying because there are three frontier LLMs producing a sizable chunk of text we read (maybe even a majority of text, for some people) lately. It would also be annoying if a clique of three humans were producing most of the text we read; we'd start to find their personal styles annoying and overdone. Even before LLMs, that was a thing that happened in some "slop" fiction genres where a particularly active author would churn out dozens of novels per year in one style (often via ghostwriters, but still with a single style and repetitive plot pattern).