Hacker News Reader

Comment by yalogin | Hacker News Reader

Without going into the technical efficacy of such schemes ( I am a skeptic), the proposed solution requires the entity generating the media to use it. Isn’t that a flaw? Why would an attacker use it willingly? If they did not want to push an AI generated content as a real one, they would have willingly made that distinction themselves.

The point is, there is no good solution here unless there is regulation, but these attempts at solutions are useful in the long run

View on HN

GenAI producers have an incentive to watermark ... it helps them avoid consuming generated output for their own training processes. Most "attackers" aren't going to be sophisticated enough to use a modified non-watermarking tool, and those are likely to fall behind in capability over time anyway. So there's a decent chance here that things could align for watermarking without needing regulation. It probably hinges on whether the stenography can be good enough to avoid being trivially removed or undone.

I disagree that attackers aren't sophisticated enough to use modified tools. There are entire work campuses dedicated to committing fraud. There's also state sponsored subterfuge. There's no reason to think that bad actors are intrinsically unsophisticated.

> GenAI producers have an incentive to watermark

Yeah in the long run you might be right, but there will be lots of people looking for a quick opportunity. E.g. a content farmer trying to SEO. If Google punishes websites for serving AI generated content, you know where this will end.

There's nothing wrong with consuming generated output for training. Blindly accepting trash input is the issue.

Did you mean to stay steganography perchance?

Putting on my suspension of disbelief goggles, I've long desired an internet with more cryptographic provenance, just some way so when people share a tweet on some other platform, I know that string was signed by that author, or some photo published by AP is signed by AP. It's just a real "we have the technology" situation. Keybase.io was moving us in this direction before they got acquihired by Zoom.

Anyway. In this future, my camera would have its jpgs signed with Nikon or Sony's certificate [0][1], art created by OpenAI would be signed up open AI, and images without any signature would be suspect by default.

[0] https://www.dpreview.com/news/6352280282/sony-content-authen...

[1] https://en.wikipedia.org/wiki/Content_Authenticity_Initiativ...

Regulation makes this worse. Imagine if every gen-AI media was required to have some signature. The bad actors won't use this signature. Or worse, they will add the signature to real media to cast doubt. Or both at the same time.

Also, how much gen AI is enough to require a watermark? If I generativley fill the part of my photo that I blurred with my thumb, should the entire photo be watermarked? Just the part that I filled in? What if I use a gen AI tool, but only apply a transformation that does not require gen AI (like adding a label on an image)?

Cryptographic watermarks for Gen AI or for non-Gen AI media is dangerous from a security standpoint.

> Why would an attacker use it willingly?

Lots of people are very lazy.

Citation: all the times we already spot obvious AI-generated comment, which in the early days included the models literally saying "As an AI language model" and this ending up in Amazon reviews etc.

Example: https://twitter.com/KingSafdar441/status/1776041379040309612

i also doubt practicality. but if the technology worked "well enough" and the major consumer-facing AI companies did it, it would help a lot with certain problems.

if you're technically sophisticated you can easily evade it of course. but 1) most people aren't technically sophisticated, 2) one can ask "why are you going through all this effort to remove the watermark if you aren't trying to deceive anyone"?

Reply on news.ycombinator.com