Scanning for Pangram Errors
veryfineprint.substack.com
veryfineprint.substack.com
That said, it did incorrectly flag something that I personally wrote as AI, which baffled me. I spent a while playing around with the text and feeding it back into Pangram, trying to figure out what the issue was. Turns out, I had a section in the text with three bullet points. I removed the bullets (literally just the bullets themselves, no actual text) and it passed as human. While it seems to perform better than anything else, I’m a bit skeptical of their claimed success rate.
On the one hand, it makes the system more brittle and arguably overfit, since it's not making a decision on the content of the writing itself. Most people would say that AI writing is still AI writing even if bullet points are removed, and vice versa.
On the other hand, we know that AI writing does has a very specific formatting signature (e.g. em-dashes, bullet points) so it seems unwise to completely ignore it.
Since AI I have noticed a lot of sudden bulleted lists appearing in pretty benign forum posts. That along with "You are right" "You are correct" "you're right" as the response.
For student work I would think it’s reasonable to look for patterns across each student’s work. A single Pangram positive across 10 assignments shouldn’t be enough to bring suspicion upon a student, but when 8 out of 10 of someone’s assignments are coming up as AI it’s time to talk.
The trend I’m hearing from my teacher friends is that students are generating AI papers and then putting them into tools like ZeroGPT and Pangram before submission. They do what you did: Remove sections, rewrite things, and make adjustments until the AI score goes down enough.
I heard that one classroom had developed a trick of adding spelling mistakes to ChatGPT output to try to make it look human written. So the teacher was getting pure AI slop but every paragraph had a couple typos inserted.
As for the 3 bullet points: That was a huge tell for one of the recent popular models. Almost all writing output would get sections of plus or minus 3 bullet points in a certain style. The newer frontier models do it less but that was a big LLM pattern for a while. Unfortunate for anyone who liked to write with 3 bullet points commonly. They’re feeling the pain of people who were emdash enjoyers before 2023
Strong disagree here. If a tool fails the basic test of teacher getting flagged on some text that they wrote personally, that tool should never be used. Being accused of cheating can have huge repercussions on a student's future. No way should a black-box classifier be used to decide their fate.
It's also bad use of statistics to claim that an indicator can never be used unless it's 100% perfect. When you start accumulating constant triggers, you do a deeper investigation.
For example, bring the student in and give them a surprise (no preparation) quiz about the paper they submitted. The students who wrote the paper can talk about what they wrote. The students who submitted ChatGPT usually can't.
You don't just put the paper into Pangram and then let it decide if the student gets punished based on the score.
People mention Pangram has a low false positive rate, but usually such statistics are over a large population. They tell me that if I check many random works I should get a low number of false positives.
But in your example you aren't checking 10 random works from an assortment of authors. You are checking 10 works from one author.
The example from the teacher suggests that something as simple as a section with 3 bullet points can lead to flagging. When you are checking 10 works from one author all it might take for them to get most of them flagged even if they are 100% human written is for that author to have some style choice, like those 3 bullet point sections, that they just use in much of their writing.
This raises the question of what a teacher should do if they talk to the kid and the kid says it was all human written. Should they do what that teacher did when his own written was falsely flagged and start figuring out what tweaks would get it go pass?
That might help if by doing that you find out that all the 8 flagged items have one simple tweak that clears them up. Then I think you have to give the kid the benefit of the doubt and act is if their style just clashed with the detector.
But if different ones take different tweaks to fix that doesn't really say much, because it seems unlikely a detector would have just a single weakness (like the 3 bullet lists) that is good at false positives.
As I mentioned it is usually error rates on random populations that get quoted at least when I see articles about AI detectors. Have any of the leading checkers published research on the distribution of error rates when checkers are used on multiple works from a single individual?
That'd be if they had a false discovery rate of 1/10,000.
If for instance:
* 100,000 samples are tested
* 100 of which are AI-generated, the rest human-written
* Pangram flags 50 of the AI-generated samples (true positives)
* Pangram also flags 10 human-written samples (false positives)
Then the FPR is 1 in 10,000, but the chance that a flagged sample isn't actually AI (FDR) is 1 in 6.
https://www.graphpad.com/guides/prism/latest/statistics/comm...
(OP isn't exactly talkigh about p-value, because it's not comparing means, but the main mathematical structure of the Bayesian error is the same)
Of course, it's unclear if the article is using the wrong term, or the wrong definition, or if Pangram is just lying in the first place.
False negatives = 50
False negative rate = FN / (TP + FN) = 50 / (50 + 50) = 50%
But as the parent says, this means you're still making plenty of mistakes. You're just not making the specific kind of mistake you're trying hard not to make, though I'm pretty skeptical of anyone claiming 1% false positives. It's hard to even envision a truly valid way of assessing this. You'd at minimum need an adversarial test set created by an independent third party that has no incentive to be favorable to you, and even then, the landscape is so constantly shifting that an error rate estimated at one time tells you not exactly nothing about a future time, but this isn't exactly estimating the strength of gravity. You can be correct one week and wrong the next.
Not a statistical estimate, but getting shot in the head doesn't even have a 99% kill rate.
They are advertising FPR<1% and FNR<1%
I'm just as skeptical as you. They mention methodology but it's pretty light.
These tools are definitely not 100% perfect, which is the primary complaint used to dismiss them. However the error rate is also getting impressively low.
In most cases I see socially and online, there is a high suspicion that the content is AI generated before someone thinks to submit it to Pangram. It’s used on-demand as a tool to confirm suspicions. I have seen several cases where Pangram had some false negatives where the content was judged to be likely human written but the author later admitted it was written by an LLM.
Pangram is very interesting in the context of Substack because the platform was a target for lazy AI newsletters. People realized they could start 10 (or maybe many more) newsletters and spend only a few minutes getting ChatGPT to write posts for them. Starting an AI generated substack and trying to get paid subscribers for it was becoming one of the popular ways to use AI to try to get a little cash. Having a tool that makes it a little bit harder, at least until the LLMs get good enough to evade it, was important for the platform.
Currently people care because so much of AI output is still poor quality. But given I personally know of someone with published AI-generated works where the reviews are gushing and positive across the board, I doubt people will care whether works are AI generated for very long vs whether the text is good.
A way of certifying a work as above a certain literary quality would likely have far more longevity, as I doubt the market of people who care whether a given work is AI or not irrespective of quality will be particularly large.
As a concrete example relating to books, the success of the Stratemeyer Syndicate in packaging up book series ghostwritten by "nameless" authors under single pseudonyms is an example. Stratemeyer series like Nancy Drew has sold more than 80 million copies (for Nancy Drew alone). At one point 98% of American children listed a Stratemeyer title as their most popular books.
How many of those do you think saw those works - most sold as a work for hire for next to nothing and churned out by the hundreds - as "an intended artistic thing"?
People who care a lot about literature tends to seriously overestimate how readers engage with writing. As it stands it's hard to get people to care enough to read, much less to get them to care about artistic intent.
I think people who bet on people caring about that as a means to stem the tide of AI content will be very disappointed.
I think this has less to do with the quality of the text and more to do with whether people have standards or not.
For every person who reads a paragraph of AI text and is instantly disgusted, there are probably 10 or more people who will gladly read pages upon pages of the sloppiest slop imaginable copy-pasted straight from chatgpt.com without batting an eye. That's just how things are, most people have very low standards.
Getting any of the current codemaxxed models to output natural text without obvious AI-isms is not an easy feat, and I doubt this person you mentioned has somehow cracked the case.
You get 80% of the way there just by giving them a writing sample and asking them to copy the style paragraph by paragraph. You get a lot further with some basic statistics to look for seriously out of distribution word use.
The hard problem isn't prose, but consistency over a long text.
I don't know how he's achieved that, but I've seen the reviews and none complained about consistency issues, which readers tend to be sticklers about.
What genre, if you don't mind me asking?
Yep, Pangram says they think false positives are way worse than false negatives, so if their AI isn't sure enough they prefer a "not ai" output to an "is ai" output.
(Somewhat-plausible-to-me explanation: It's looking for various stylistic features; older writing very rarely has the most AI-like features, or perhaps almost always has some non-AI-like features that outweigh whatever signs of AI-ness might be there. Present-day writers are more likely to get wrongly flagged as AI.)
Right now many niche book communities are facing an onslaught of unlabeled AI works, mine included.
The second read was deliberate, and motivated by trying to understand my initial confusion. I considered the sentence structure and realized that 'mine included' could, if you ignore strict grammar rules, refer to 'communities' (which made more sense in context) and not 'works'.
Those skills are meant for humans, not for models. Pangram said in their recent model update that they now also detect the humanizers.
All it requires on their end is running each of their AI outputs through each humanizer and including it in their corpus as ai-humanized.
It's essentially impossible now to prompt your way to non AI detectable text. Even if you do today, it'll be picked up by the next model update.
This is why in academic environments its probably worth re-testing old exams or papers periodically - same way blood and urine samples from athletes are preserved to take advantage of better future testing.
Now imagine if they put the same effort into detecting if a given text was produced by one of a few dozen super-prolific human writers. I'd imagine they'd get pretty good at that too.
The main limitation of those "humanizer" rewriters is that most of them focus on making the next less detectable to humans, by making them read better. There's likely to be plenty of signal left that isn't affected by trying to make the text read better the same way human writers have plenty of idiosyncrasies despite being human.
[0] https://archive.org/search?query=analog%20science%20fiction%...
1. P(actually true | predicted true) (= true positive rate, recall, sensitivity)
2. P(actually false | predicted false) (= true negative rate, specificity)
3. P(predicted true | actually true) (= positive predictive value, precision)
4. P(predicted false | actually false) (= negative predictive value)
If you just try to minimize the share of false positives, you are only maximizing 1.
The measure which maximizes all four values is the binary Pearson correlation, also called "phi coefficient" or "MCC". It is 1 if the classifier is always correct, 0 if it guesses randomly (its predictions are statistically independent of the facts) and -1 if it always predicts the opposite of the true value.
https://www.pangram.com/research/model-card/pangram-3-3
Academic Writing FNR=0.00% N=48443
Creative Writing FNR=0.23% N=41940
RAID FNR=0.93% N=66855
https://www.pangram.com/solutions/chrome-extension
(disclaimer, i am a pangram research scientist)
My experience with Pangram so far in Substack is nothing short of phenomenal.