Hacker News Reader

Comment by MichaelDickens | Hacker News Reader

Pangram says "100% AI-generated". I trust Pangram more than I trust this author's word.

Pangram is a complete scam. There is no durable way to detect whether something was written with AI. I've had folks say they ran pangram on my writing and it was "100% AI". Except I don't use AI to write or edit anything I post publicly.

Can you give some examples of your writing that trigger "100% AI" on pangram? I'm very interested in studying pangram's false positives. Especially if it's something published before ~2024

Not my article, but this is quite convincing of the brittle nature of these tools.

https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...

I agree Pangram's UI is awful and often misleading. Their underlying classifier model is pretty accurate in my experience, though, at least in the sense that it has very few natural false positives. If you disagree, please send me some long-form verifiable false positives that were not explicitly written to trick Pangram. I love to learn.

I just sent you a long article which you could not have read in the time it took you to reply. I would suggest you start there and read that article, which outlines several cases where the author was able to create contrived false positives and negatives.

I read this article about a week ago. Did you read my reply? I am interested in natural false positives (ones not written to fool the classifier.)

Contrived false positives and negatives could (and should!) always be possible. That doesn't tell us anything about the natural false positive and negative rates, and a lot of people who have actively tried to use voice instruction to get models to consistently fool the detector without iterating against it directly have failed.

Please try https://news.ycombinator.com/item?id=49212788

I'm sure you'd like $100 then to give the founder of pangram an essay they classify as AI that is in fact not ai.

Free money, right?

https://x.com/max_spero_/status/2085041200923394459

Paul Graham mentioned Pangram the other day.

I went to the site and pasted something into the text box. The next step was to sign up.

I don't trust Pangram.

Those people should just try posting a big quote from the article to HN, and then see their comment get instantly flagged due to HN's AI detection (which might as well be Pangram)

It's baffling to me how people are so dismissive of Pangram despite never using it, extrapolating their experience from GPTZero or something else.

Saying "more" seems generous. I don't trust Pangram at all, and I also don't trust the author at all. Both can be trustless charletans simultaneously, and I feel this speaks volumes about the state of digital media today.

So you would trust a known writers word over non deterministic software ? I read New Orleans will soon let AI handle 911 calls, Hope the AI believes the calls are from humans :)

Well, that is a shame, in my opinion. I trust humans significantly more, and I hope that never changes.

Here's another AI detector I just built:

  text.indexOf("—") > -1

Holy shirt balls, they had AI in the 1800s! Fricken' Dickens was a bot, and Austen was a clanker.

You know, ever since em-dashes blew up due to their use as LLMs detectors I've been using them more and more in my writing. They're great!

I get the market need for AI detection, but the fundamentals of it are so woo-woo that I despair

Pangram said my 100% AI generated cover letter was "57% AI assisted." And the rest was human. lmao

That's idiotic, frankly.

Are you serious? Pangram is notorios for false positives!

Can you give some examples of Pangram false positives? Ideally ones from before 2024, or otherwise ones from notable writers who started writing before 2024.

All ML classifiers (and algorithms) have a non-zero false positive rate. Having an error rate is baked into every ML classifier and algorithm. And its always non-zero in practice. In fact, hitting every test in some sort of test suite is likely a sign of a less accurate classifier, not a more accurate one.

Well no shit, but that doesn't mean you just trust them just because they say it works. The cost of a false accusation can be very high.

oh you trust pangram. ok i now trust pangram....

lmfao.. what were you hoping to astroturf with that?

Trusting snake oil is a wild choice.

Reply on news.ycombinator.com