Hacker News Reader

Comment by simon84 | Hacker News Reader

The goal of the AI act is not to determine if an "oh yeah!" comment was AI generated. The target is long papers that falsely claim human review and can have real significant consequences.

E.g. research paper, law makers, lawyers, state policies, notaries,...

These are much longer content and thus statistically they will disclose a better guess at AI generated content.

Asking another AI to paraphrase will not erase the mark (which they are unaware about) but rather cumulatively add their own mark and make it easier to detect.

The problem is not to use AI, but to endorse the responsibility of the content you (as a human) deliver and somehow make sure that fake-news, biased content or unverified output is detected as early as possible.

View on HN

You'd think that the humans being paid to review these things are actually reviewing them. Journals, laywers etc. are expensive. With the addition of AI, it should be easier than ever to review things on their merits.

Maybe some part of it is that the deluge of slop is uncovering how poorly/sloppily these social institutions were working in the first place.

I think you are very correct here! Now imagine how much worse it can get with AI in the way.

It is as always (think cybersecurity) the cat and mouse game, what is a weapon is also a defense. You, the simple fact that you are on this very site, means that you are probably more educated to AI than most, so it is not necessarily you that will really benefit from any constraining framework.

There are many people believing in many weird theories and those are more prone to be convinced by a nice narrative. AI did not introduce that, it just made it easier and available to anyone with any intent.

Peer reviews is largely done for free. In some cases, an author needs to commit to reviewing someone else's work for their work to be reviewed. However, this has led to an increase in LLM usage for review generation, even if conference/journal guidelines forbid it. I've seen nonsensical reviews from people who obviously haven't read the paper beyond an LLM-generated summary. Area chairs are supposed to catch this, but I imagine they're using LLMs too.

> With the addition of AI, it should be easier than ever to review things on their merits.

It makes it harder, because amount of bullshit goes up and it is easier to generate plausibly sounding bullshitm

I don’t understand how asking another AI to paraphrase causes both watermarks to remain

From what I understand of the not-very-detailed text watermark, it comes up to masking the generated output with specific bias in the weights of the token generation (or something like this). Anthropic was explaining that it would survive edits (not full rewrite). So asking another AI to paraphrase will surely add its own, but since the bias is not disclosed, it will depend on how the 2nd AI is considering the existing tokens to steer its own.

Fictive example: imagine the bias is to inject notion of colorfullness in the text regardless or its content:

"I eat an apple" becomes "I eat a red apple"

The 2nd AI is about injecting size component, so the text becomes " I eat a big red apple"

There you get both watermarks. It is not that obvious obviously

Reply on news.ycombinator.com