Identifying Stable Diffusion XL 1.0 images from VAE artifacts (2023) | Hacker News Reader

Identifying Stable Diffusion XL 1.0 images from VAE artifacts (2023)

hforsten.com

68 pointsrcarmo24 comments

Generally, the VAE is mapping from a small latent space to a large image space. This means that there must be a large number of images for which no reverse mapping exists.

It should be possible to identify images that have not been generate by the VAE since they are not part of the set images that the VAE can generate. The other way round is a bit more difficult as there may be images that can be mapped to the latent space and back without loss but have been generated in another way

-> there may be false positives.

This logic has a key flaw: just the fact that the size of the space is different doesnt mean that every representable thing in the larger space is a thing we care about. E.g. a person with three hands may not have a representation in the smaller space, but we would never care about that. The actual question is: what is the difference in the amount of information encoded in a large image vs the small latent space and compare that to the difference in information between a large image and a small image. If those two differences are close enough together, being able to determine a legitimate difference between SD generated vs not becomes near impossible.

Yes, otherwise cryptographic hashes won't work (they are not bijective)

The logic is still the same. If the VAE is trained so that it is biased toward human preference, then the probability of false positives in real world images would increase.

8 months ago StabilityAI changed the default VAE from 1.0 to 0.9, so no one realistically uses VAE 1.0.

And this article is nine months old.

I can see why, that VAE looks pretty bad.

Has anyone tried training a neural net to distinguish between photographed vs AI generated images?

Of course, you will need to remove exifs or other metadata, but this sounds like the kind of domain that NNs are good at.

yes, and it's quite challenging for instance => https://sightengine.com/detect-ai-generated-images

Isn't that basically how the image generation models themselves work? By refining the image until it can't distinguish it from a "real image".

That was trendy for a while but is no longer the primary method

Yes. Trying to sell an "AI detector" is a fool's errand since this is how adversarial networks (the hot model from two hype cycles ago) are trained. The only use of the "AI detector " is to tune the model so that the detector is uniformly random.

This is true for explicitly adversarial actors. I can imagine a serious-business misinformation group investing the money required to train up a generator designed to defeat automatic detection.

For everyone else though, the people making common tools everyone uses explicitly want their images to be easy to automatically ID as fake. And, the users largely prefer it that way too.

As a user, I don't really care one way or the other. I'm not trying to trick anyone. I also don't care if they can readily tell if my image was AI generated. I don't post much media though. If I did, I'd probably use it for a game. And if it's a game, does it matter if I touched it up in Photoshop vs StableDiffusion? There's going to be some processing either way to get the art asset I need.

The toolmakers might have a mild incentive to tag images generated with their tool to prevent too easily creating these 'fakes' or just for analytics purposes so they can see how their output is spreading.

But regardless, it's a fools errand like the other poster said. Anyone that is serious about tricking the mass media will strip out the watermark or use a different tool.

It's 2024. Everyone is an adversarial actor.

What was the hype-cycle between GANs and diffusion?

Cryptocurrency.

If you have a scoring system that tells you how "AI-like" a picture is... congrats, you built a loss function that can be used to train AI.

Telling how "AI-like" an image is is done by seeing how well it completes a round-trip through the VAE.

It will either match a known VAE well, or be an image that's altered enough (jpeg noise, photoshopping in post, etc) to not match the output of a VAE.

And meanwhile the "loss system" for training AI idea is still hobbled by how the image generating AI systems are all using VAEs. At best, you've swapped one VAE for another. Just another new VAE to add to the list of VAEs to check.

And the next round of spy vs spy begin... AI generated content vs detecting it... just like the age old email spam problem.

Very interesting! Well broken down and explained!

It falls short because you can just transpose the latents and use whatever vae you want.

(2023)

Added. Thanks!

Reply on news.ycombinator.com