Do you regularly find text content that you know is AI written (but is not marked as such)? Because honestly I don't, and it must exist in decent quantity by now. Or perhaps it's still sparse?
[1] https://en.wikipedia.org/wiki/Wikipedia%3AAI_or_not_quiz [2] https://en.wikipedia.org/wiki/Wikipedia%3ASigns_of_AI_writin...
AI generated. Some of the clues include:
- Most obviously, a failed ISBN checksum
- Other source-to-text integrity issues; for example, the WWF source says very little about Malaysia specifically, only mentions Sunda tigers (Panthera tigris sondaica), and does not mention tapirs at all
- Very short yet consistent paragraph length
- Generic "see also" links, one of which is redlinked
This is not the sort of thing that I pay attention to unless I'm doing detailed research. And even then I'd probably have a bot check these for me, ironically, since it's such a mechanical job. At the very least detecting AI like this requires conscious effort.
I can easily tell AI writing. I'm sure plenty goes under the radar, but I can still catch a lot.
The gap between LLM-generated writing and the composite style of the average Wikipedia page is more narrow than most people may believe.
The more you see those patterns the more you start recognizing them. By now I can recognize quickly if a blog post or README.md was generated by Claude or ChatGPT because the signs are so obvious.
Even Hacker News comments that are AI written are easy to spot if they weren't edited. I know I'm not alone because when I recognize an AI comment I check their comment history and find other people calling out their AI-generated submissions, too.
Learning how to recognize the output of the popular AI models is becoming a critical business skill, too. You need to be able to separate out the content from someone who was doing real work that you should take seriously as opposed to the output of someone who is having ChatGPT produce volumes of text that they don't review. The people who do that will waste your time.
Ask it to write in the style of patio11 or someone else with a distinctive tone, and it will do a remarkable job.
It will pass pretty consistently. Not sure I love it.
However the default tone and output style of Claude and ChatGPT are very obvious.
> My recollection is whenever I've seen data on this--the educators who think they can spot students cheating--the conclusion is people are really bad at identifying LLM-generated content.
If you can share that data we can discuss it, but there's nothing really to discuss here without a source
Among people who review a lot of user-submitted content, it becomes easy to spot the consistent voice of LLM writing. Wikipedia has a full page on it: https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
This article doesn't have the tells, it looks human written.
It's possible I should envy you, I'm not sure.
I can't prove it but I'm comfortable enough in my judgment to say it.
HN and YouTube are the worst offenders for me.
I think those who are very opposed to AI often don't know much about the real limitations since they don't use it, and their complaints are often a year or more out of date.
I think the ideal demographic for spotting these are people who use the frontier LLMs a lot and they also have worked with text in detail, such as copywriters, people who have learned foreign languages and grammar etc., have edited articles for language and generally have a more "wordsmith" look at language and are sensitive to flow and rhythm of language on a more technical level.