ZipPy: Detect AI-generated text quickly via compression ratios

I think it'll be difficult or perhaps even practically impossible to detect if a single blob of text is generated by AI in the future. However I think we'll find there's a practical means of determining if multiple blobs of text in a set of blobs was generated by AI. For example, if you have a class of students turning in reports on the same topic: an AI would produce similar outputs for roughly the same prompts because it was trained in a specific way. Those reports would have some statistical similarities.

But this isn't an AI detection system, it's a general cheating detection system. I'd expect it to also detect if one person wrote the reports for multiple students and simply rephrased each report. But if the intention is to detect cheating, that's not a bad thing.

This is clever but it feels like a situation where Goodhart's Law (when a measure becomes a target, it ceases to be a good measure) will apply. Once this becomes a known test, AI text generators can optimize for it and then it won't be a reliable test anymore.

For student reports, you could potentially whip something up to pull out each paragraph and evaluate it - giving you a form of score to investigate further. When you're marking 200-400 reports and you only have 5-10 minutes to mark each, something like this could be a game changer.