maxspero

> Sounds promising, right? I spent some time trying [perplexity], but results were disappointing—plenty of false positives and false negatives, and no reasonable threshold could be set.

Perplexity was widely considered SOTA in 2022. One part of it is because everyone was evaluating on open models or closed models that were still close (i.e. GPT-2 vs. GPT-3.5). Today, the gap is so much wider between the models you can use to compute perplexity and the frontier models people actually use.

Also so many AI text detection papers used a strawman RoBERTa baseline that was very undertrained for the task.

The synthetic mirrors method for data generation used here is the same as what we use at Pangram. Good blog post, thank you for sharing!

Update: this is actually ZeroBounce’s Verify+ feature which we figured out after some escalation. It’s now disabled!

Follow-up: our vendors have told us that they do not send any emails as part of the validation process. Either somebody is lying, or there's something even weirder going on. We still have more tests to run to isolate which software package it could be.

Hey! Founder of Pangram here. We use Zerobounce and CustomerIO for email validation. I had no idea this was happening. Not entirely sure which one this is coming from, but this is not intentional on our part. Will dig deeper and eliminate the part of the stack that is sending spam — definitely not good that this is happening.

Anecdotally people are seeing a rise of low-quality reviews which is correlated with increased reviewer workload and and AI tools giving reviews an easy way out. I don't know of any studies quantifying review quality, but I would recommend checking the Peer Review Congress program from past years.

It's definitely going to be a back and forth - model providers like OpenAI want their LLMs to sound human-like. But this is the battle we signed up for, and we think we're more nimble and can iterate faster to stay one step ahead of the model providers.

Pangram is trained on this task as well to add additional signal during training, but it's only ~90% accurate so we don't show the prediction in public-facing results

Thanks, fixed.

Yeah, Pangram does not provide any concrete proof, but it confirms many people's suspicions about their reviews. But it does flag reviews for a human to take a closer look and see if the review is flawed, low-effort, or contains major hallucinations.

There are dozens of first generation AI detectors and they all suck. I'm not going to defend them. Most of them use perplexity based methods, which is a decent separators of AI and human text (80-90%) but has flaws that can't be overcome and high FPRs on ESL text.

https://www.pangram.com/blog/why-perplexity-and-burstiness-f...

Pangram is fundamentally different technology, it's a large deep learning based model that is trained on hundreds of millions of human and AI examples. Some people see a dozen failed attempts at a problem as proof that the problem is impossible, but I would like to remind you that basically every major and minor technology was preceded by failed attempts.

Our benchmarks of public datasets put our FPR roughly around 1 in 10,000. https://www.pangram.com/blog/all-about-false-positives-in-ai...

Find me a clean public dataset with no AI involvement and I will be happy to report Pangram's false positive rate on it.

I am not sure if you are familiar with Pangram (co-founder here) but we are a group of research scientists who have made significant progress in this problem space. If your mental model of AI detectors is still GPTZero or the ones that say the declaration of independence is AI, then you probably haven't seen how much better they've gotten.

This paper by economists from the University of Chicago economists found zero false positives of 1,992 human-written documents and over 99% recall in detecting AI documents. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5407424

Co-founder of Pangram here. Our false positive rate is typically around 1 in 10,000. https://www.pangram.com/blog/all-about-false-positives-in-ai....

We also wanted to quantify our EditLens model's FPR on the same domain, so we ran all of ICLR's 2022 reviews. Of 10,202 reviews, Pangram marked 10,190 as fully human, 10 as lightly AI-edited, 1 as moderately AI-edited, 1 as heavily AI-edited, and none as fully AI-generated.

That's ~1 in 1k FPR for light AI edits, 1 in 10k FPR for heavy AI edits.

I've been using Grapevine at my company for the last couple weeks. One of the coolest features is that it proactively answers questions (with citations!). Not everyone thinks to tag the bot but it often surfaces the relevant answer and document and saves everyone some time.

We benchmark on pre-2023 datasets of O(10M) documents not in our training set. Other detectors seem to have between 1-3% false positive rate and ours is around 1 in 10,000 as of our latest model update. We do a lot of active learning + core set selection to keep FPR low and improve recall on larger LLMs. Our white paper with some methodology is here: https://arxiv.org/abs/2402.14873

Hey it's me, Max. I ran the analysis for WIRED and got them their initial 47% number for AI content.

The CEO accused me of trying to extort him because I sent a short email with our findings prior to the publication of this article.

He also casts doubt on the efficacy of AI detection (fair) but I think our AI detection model is orders of magnitude more accurate than the others and I stand by its predictions.

Thanks for trying it out. It's in our roadmap to expand to technical writing (currently trained mostly on creative writing). Hopefully this will fix the wikipedia issue.

I've benchmarked against Originality.ai, gptzero.me, zerogpt, writer.com and copyleaks.com, which are the top 5 AI detectors to my understanding.

None of them are very good, so I don't think this claim is very outlandish.

Also, are you sure it's not reliable or maintainable? Obviously you can't publish one model and expect it to work forever but we have pipelines to continuously augment our training set and we can add new LLMs as they come out.

Thanks for trying it out. Shorter texts with fewer sentences are certainly a challenge - they just have a lot less signal.

I tried your prompt asking for ten sentences and got 99.4%. Possibly there needs to be some sort of gate on how much text we accept before we can provide an answer.

> Talk about JavaScript in ten sentences. Use human like words instead of professional tone.

``` JavaScript is like the magic wand that makes websites come alive, turning them from static pages to interactive wonders. Originally, it was made to add some pizzazz to web pages, but now it's super powerful and does way more. It’s not Java, even though the names sound alike; think of them as distant cousins rather than twins. Browsers love JavaScript! They have built-in engines to run it, making our web experience fun. You can find JavaScript not just on websites but also in things like mobile apps and even some robots. There's this cool toolkit called Node.js that lets JavaScript play outside of the browser, giving it even more playgrounds. Developers often use libraries, like jQuery or React, to give them a head start and make things snazzier without reinventing the wheel. JavaScript can be both your best friend and a tricky beast; it's easy to start with but can get complex as you dive deeper. The community is massive, so if you ever get stuck, there are tons of helpful souls out there ready to lend a hand. At the end of the day, JavaScript is all about creating, innovating, and bringing ideas to life on the web. ```

Nice to hear of someone else trying this. Did you find any good ways to reliably trick these?

What do you mean "it won't work long term"? My opinion is RLHF and fine tuning outputs for safety and politeness ends up watermarking output in a way that's pretty reliably detectable. I don't see these going away any time soon, at least for mass-market AI products.

I don't think it's possible to determine provenance with 100% accuracy, but I think ChatGPT essentially "watermarks" itself with its RLHF, making it more polite and giving its output a very distinctive voice. ChatGPT also tends to use passive voice and generic adjectives much more often than real human writers.

Have you tried it?

Interesting. In my experience, ChatGPT always says "As an AI language model..." or lately just "Sorry, I can't help with that." Have you seen "As a large language model..." come out of any of the big LLMs?

We're trained on real ChatGPT data so am interested in hearing your prompts that result in this.

Thanks!

Can you share the paragraph you wrote?

In my experience TurnItIn's AI detection does not perform very well. Regardless this is an issue with educating the teacher - 27% does not mean the text is 27% AI-generated.