Hacker News Reader

Comment by p1esk | Hacker News Reader

“Images generated with our code use the invisible-watermark library to embed an invisible watermark into the model output. We also provide a script to easily detect that watermark.”

Interesting - if it can be detected then it can be removed, right?

View on HN

It's open source. It can be removed directly from the code.

Also, most people use other tools like web-ui to run these models, which don't seem to apply any watermarks.

Sure, I just don’t understand why they’re doing it.

In theory they could also train the model to always add a watermark.

Stability, Midjourney, Dall-e, etc don’t want their products to be used for misinformation. They really do want AI generated images to be trivially identifiable.

There are papers around describing how to bake a watermark into a model. Don’t know if anyone is doing that yet.

And many will start working on techniques to unbake it from the model. A cat and mouse game, like all other attempts to track.

Sure, but that can be said about any security measure. If we agree that passing AI generated art as human-made or as a photograph is bad, then making that harder to do is worthwhile.

We don't agree.

Why don't you agree?

Given the models’ propensity to being able to fairly-regularly reproduce gettyimages watermarks, I’d imagine putting a mark in the same place, at the same size on all the training images would train it that the mark must be there.

One reason is so that they can detect generated images and exclude them from future training sets.

There have been some talk of various countries requiring AI images be watermarked. They may be trying to signal their willingness to comply, and hopefully ensure they are not outright banned. That fact it can be bypassed isn't necessarily their problem to fix.

Many good reasons have been presented here. And on the other side, is there any reason not to do it?

(part from the effort required to put it in place)

I guess it's a form of tracking and we don't like that

There is nothing personally identifiable in the watermark though, no more than other image metadata.

And yeah, you can see the watermarking code in the huggingface pipeline now. Its pretty simple, and theres really no reason to disable it.

What exactly is in the watermark? Other image metadata is commonly used to track people, or used in combination with a few other hints to track people.

If it's strictly a primitive watermark with absolutely nothing unique per mark, then it might be no big deal.

It is. See for yourself:

https://github.com/huggingface/diffusers/blob/2367c1b9fa3126...

I see no reason to disable it. Its fast, and its no more personal than basic metadata like image format and resolution... I can't think of any use case for disabling it other than passing AI generated images as authentic.

marked images can be excluded from training data in the future, so that it doesn't eat its own output, which is known to cause model quality to degenerate (an issue likely to plague training LLMs on post-2022 datasets.)

as for training the model to add a watermark, that's an interesting idea but I'm not sure it's feasible/not trivial to circumvent.

We’ll almost certainly see models trained to produce watermarks directly in the next few months. A highly diffuse statistical abnormality would be nearly impossible to detect or remove.

And as soon as such models appear there will appear methods to process images destroying such “statistical anomalies” without changing visible appearance.

Would this not be trivially defeatable by something like a single pass of img2img with a very low denoise?

Microstates vs macrostates. Doesn't it depend on the basis for the encoding and how many bits it is? Watermark, we usually think of something in the pixel plane, but a watermark can be in the frequency spectrum, and other steganographic (clever) resilient basises.

Simply running a cycle of "ok" quality jpeg compression can completely devastate information encoded at higher frequencies.

Quantization & subsampling do a hell of a job at getting "unwanted" information gone. If a human cannot perceive the watermark, then processes that aggressively approach this threshold of perception will potentially succeed at removing it.

At this point the watermark can be meaningful content like "every time there's a bird there is also a cirrus cloud", or "blades of grass lean slightly further to the left than a natural distribution".

Because our main interest is this meaningful content, it will be harder to scrub from the image.

That would be indistinguishable from a model that was also trained on that output, wouldn't it?

It seems much more likely that it's their solution to detect and filter AI images from being used in their training corpus - kind of a latent "robots.txt".

Different tools have different levels of ability to remove invisible data, e.g. jpegli vs. libjpeg-turbo

Depends on the watermark. The current watermark uses a DCT to add a bit of undetectable noise around the edges. It survives resizing but tends to fail after rotation or cropping.

A robust rotationally invariant repeating-but-not-obviously micro watermark would be quite a neat trick.

An orthogonally rotational invariant watermark would cover 90%+ of real world image transforms - think flips, mirrors, and 90 degree rotations - and would probably be much easier to create.

Watermarks can be made to be pretty robust, though having an open source detection routine will make it much easier to remove (if nothing else, if you can differentiate the detection algorithm then you can basically just optimise it away). The kind of watermarking that is e.g. used for 'traitor tracking' tends to rely on at least a symmetric key being secret, if not the techniques used as a whole (which can include more specific changes to the content that are carefully varied between different receivers of the watermarked information, like slightly different cuts in a film, for example).

Reply on news.ycombinator.com