Comment by esperent | Hacker News Reader

Comment by esperent | Hacker News Reader

Parent Full thread

> Based on some of the examples they explicitly provided, it is clear to me Stable Diffusion creates novel art.

You're jumping to a conclusion that the data doesn't warrant, I suspect because it's a conclusion that suits you.

They're doing something like a "reverse image search" from an AI generated image over the original dataset and returning a few examples that have a high degree of similarity. There's no guarantee that the images they return are actually the ones combined to create the AI image. In fact their results are dubious - see Saurik's comment:

https://news.ycombinator.com/item?id=34671149

I don't have an opinion regarding the question of whether SD creates art or not. To be honest, as someone who enjoys art or kind-of-art in many forms, it's not important to me (although, of course, I can see why it is important to someone who makes a living from art or AI generated images).

However, this website doesn't add anything except noise to either side of the debate, from what I can see.

Using pixel-space or latent-space distances to measure if a generative model has simply memorized the training data is a common evaluation metric in ML literature. If the website is using the full training set of LAION 5-B used to train Stable Diffusion, then I find it extremely convincing that SD has generalized to the data distribution it was trained on, and does not simply regurgitate other people's artwork. If there is a way to find more similar images, I have yet to see it.

I think this website is evidence SD is a novel image generator, and rarely creates infringing images (at least in the way we thought of infringing before these kinds of AI).

Until somebody comes out with a better way to trace back output images to training data, this is the best "data" we have so far.

Before this, there were anecdotal examples of SD outputting an image with some Shutterstock watermark or very similar to some artist's work, but the prompts also seemed highly specific or were asking for something in that artist's style.

This tool at least lets us start to trace back the average image, and so far it does seem SD is adding something novel to its outputs.

> Until somebody comes out with a better way to trace back output images to training data, this is the best "data" we have so far.

There are many better ways, in the sense that they actually do something like estimate the causal effect of a specific training datapoint, like leave-one-out cross-validation training or surrogates to Shapley value, or using nonparametric models to trace backwards. This is a whole subfield of ML research.* (The primary summary is: "it's hard and the easy approaches don't work." Which why he's not doing any of those but an easy incorrect thing.)

* I'm not entirely sure why anyone cared so much... Research topics can be kinda arbitrary. But in this case, I think there was something of a fad around 2017 that there were going to be 'data marketplaces' where you would be trying to estimate the value of each datapoint to price it. This turned out to not exist as a business model: you either used big public data for free for generic model capabilities, or you had small proprietary data you'd die rather than sell to a competitor.

I have to be correcting you here, stable diffusion does not combine images.

Reply on news.ycombinator.com