Hacker News Reader

Comment by Oarch | Hacker News Reader

This error right at the start initially made me suspicious that the article was generated: "in a way that has never before detected before"

View on HN

I make errors like that all the time when I write a sentence out, realize it reads awkwardly and then edit it incompletely, eg start with:

"In a way that has never before been detected"

(Sound it out in my head, sounds awkward... Decide it would read better as "in a way that has never been detected before" but edit it non-linearly to reduce the amount of cursor movement)

First change:

"In a way that has never before been detected [add: before]"

Second change:

"In a way that has never before [remove: been] detected before"

Oops, I meant to highlight the first 'before' and hit delete but I mistakenly highlighted 'been', probably because my lazy brain was satisfied with seeing 'be' while searching for before.

Hopefully I catch that on the 2nd edit, but if I'm in a rush maybe I don't.

Honestly, if you are using one of the most recent LLMs, you do not get errors like that. In many ways typos or editing mistakes (like I suspect this is) are signs that a human wrote it, not an LLM.

They make errors like that if you tell them to make errors like that.

"Ensure your response contains at least 5 grammatical and 3 typographic errors"

that is the type of error humans make frequently but LLMs make rarely

So, the fair assumption would be to assume all content with too good to be true writing and grammar is AI written?

No, because many writers have editors whose role is to prevent mistakes like these from going to print

So, if bad grammar -> human. If good grammar -> could be both?

I don't think there's any sure way to tell, and trying to do so in the past has massively disadvantaged people who have English as a second language & get accused of plagiarism via LLM (can't find source with a quick google, but there was an ESL student accused of plagiarism by an 'ai text detection' model).

I think we just have to embrace the fact that any text could be wholly/partially LLM generated. An LLM could be asked to introduce errors (or a human could edit the output, either to purposefully introduce errors, or naturally introduce errors as part of their process). We should focus on:

1. Developing our skills to verify people's claims (which has always been important)

2. Trying to understand the motivations/agenda behind the text (which I reckon is even more important still)

The above two steps insulate you from a wide range of propaganda/trickery, and are the sorts of things we should have been focusing on in schools for decades already.

Mentioned elsewhere, bad grammar could be a human telling an AI to be plausibly bad.

Perhaps not yet, but surely a generation from now.

At first I was suspicious at first.

Did you notice any other clues? I noticed that too and just attributed it to sloppy editing, but you might be closer to the truth.

In this case an early version of the AFP newswire article appears to be at fault; it's just that the claim of "proofread" (in "Editor's notes", top left above headline on phys.org) is perhaps somewhat exaggerated.

One can see the same "never before detected before" error on numerous other aggregators: <https://duckduckgo.com/?q=%22never+before+detected+before%22...>.

I wonder if that argues more to AI vs. Human "editing." It seems to me that an AI would be less likely to catch that when it's composing a story from sources in the Internet.

Reply on news.ycombinator.com