Comment by Barbing | Hacker News Reader

Comment by Barbing | Hacker News Reader

Parent Full thread

Recent Pangram feedback on HN:

>Have you tried putting known human writing into pangram? I have. I've gotten 100% AI with multiple samples of my own human writing. It has also given me 50% on things I know were 100% AI written (from my prompts).

https://news.ycombinator.com/item?id=48326698

>Pangram is basically a made-up number. / I've tried it on large docs I've written well before the AI times, and that are nowhere available on the Internet (so it can't be a corpus issue) - and it is happily classifying me as 60%-80% AI.

https://news.ycombinator.com/item?id=48378226

Two of my own thoughts:

Unfortunate there's an incentive to pay to sign up to protect oneself against false accusations.

An earlier claim in this thread stated 100% from the same tool, but another commenter claims 76%, so apparently the tool is even susceptible to that failure mode.

Skepticism against AI text detectors on HN is as old as time, and frequently comes from people with some vested interest in filling up the internet with slop when you look at their business ideas / projects / blogs. I've done systematic testing on human and LLM-generated text and I'm confident that the accuracy is higher than 95% (and that <5% is almost exclusively false negatives).

You shouldn't crucify people based on this alone, but if it reads like AI, quacks like AI, and is detected as AI, it's probably AI.

>Skepticism against AI text detectors on HN is as old as time

Since early 2023 or so, when the detectors were widely reported (off platform) as unreliable?

>and frequently comes from people with some vested interest in filling up the internet with slop

I'm sure sometimes.

At least once, has come from someone who recently philosophized about whether to call out the specifics of AI writing in the first place given the potential for aiding labs in their training missions https://news.ycombinator.com/item?id=48326913

>I've done systematic testing

You in the field?

>but if it reads like AI

Actually didn't to me, and I'd like to think my detector's no worse than the average for a commenter here... perhaps as we'd all :)

Reply on news.ycombinator.com