Another bunch of dead give aways in code bases with READMEs is the repetitive:
- "No X, No Y, No Z." pattern
- "Here is X - it makes Y"
The worst and most obvious one is the constant over use of emoji ticks and crosses.
- "No X, No Y, No Z." pattern
- "Here is X - it makes Y"
The worst and most obvious one is the constant over use of emoji ticks and crosses.
*actually a hyphen but it's functioning as an em dash.
This reminds me of another em dash+AI related topic: I've noticed LLMs have an extreme bias towards spaces around the dash while people can go either way with it.
A human writer would add that sparingly, for cases where there might be some ambiguity. But the bot added that to basically every single API endpoint.
I put to you, if you see a trope in AI writing it's because that trope appeared in the training corpus. Therefore, sure, being predjudice against it lets you catch some AI, but you'll also flag human outout. I think that may not be worth it in the end.
There are many cases out there of people proving that it does happen. I have personal text not published that do trigger all detectors between 70-100% AI depending on the tool. I wont be sharing these, as "providers" of these "tools" would simply add it to training data and continue to merrily overfit.
Bottom line, transformers regress to the mean like many other models, if you as a person produce output aligned with mean of corpus, you'll trigger detection. More importantly, evading detectors is trivial. Find a corpus of text from an author, get an llm to write a note on style, parlance, habits in writing etc. and then use that voice file to drive outputs from an llm. If the source text didnt register as AI, the new ai output also reliably avoids detection.
So my problem is, detection doesnt work, false positives are fact, so these tools at best offer harm.
A bigger problem¸ undermining your request for proof beyond that which many before me, and including me, burned in an effort to make folks see reason, is that even if i handed you proof on a silver platter you wouldnt understand it. If you could, you'd already understand, because the problem and the math are quite simple. Every example text i've offered in the past now registers 100% human, but many I kept to myself continue to show the same problem, and that never changed. So why would I waste my few remaining tools for sanity checking, when all i can expect from that endevour is losing a tool and shifting nothing in the conversation? No, best I keep that to myself for now.