Hacker News Reader

Comment by lifthrasiir | Hacker News Reader

> Many clearly think it's a photocopier. As if StableDiffusion is hiding copies of all these works inside.

In my latest series of tweets [1] I tried to argue that this is not the case in a less technical way, not sure if I was successful though. For anyone not fluent in Korean, my argument was as follows:

Stable Diffusion was trained with a data set composed of about 2.32 billion text-image pairs. The model itself by comparison weighs about 4 GB. If the model somehow stored those images (completely disregarding texts) in the compressed form, it would be less than 2 bytes (EDITED) per each image. Whoever downloaded an image from the internet can see this is absurd. In reality 4 GB of data is the juice extracted from the data set, and if you could actually recover an original image from it then the juice must have had a recognizable residue, which would be pretty undesirable.

[1] https://twitter.com/senokay/status/1565543214068600832

View on HN

Not a good argument, because this type of AI is basically exactly a compression algorithm - it’s the most efficient way we know to compress 2 billion image/text pairs down such that it can recreate the average of all the inputs retaining as much layered knowledge as it can.

It’s a very very fancy compression algorithm, but it still is one.

There is an important distinction between lossy compression and lossless compression. My information-theoretic argument shows that models couldn't have been losslessly compressed, and the juice analogy implies (but does not show) that there is no obvious way to "decompress" the original data set. I intended that, if both arguments are true, those models can't be a photocopier because they don't contain copiable works at all.

The law as far as I know is concerned with derivation not exact copy.

It’s easy for these models to produce very close images to the source material with the right short input.

The model size:training image ratio seems more like an implementation detail more than anything.

The fact that people are writing prompts with named artists, and phrases such as "Trending on Artstation"- yeah, I'm not sure if you can just handwave that away.

“Trending on Artstation” is a genre curated just like Bauhaus or Romanticism. Just the wider populace votes instead of old noble families and wealthy bankers.

I don’t see the moral significance of the former genres versus the latter

In some ways it’s just a coincidence that those prompts do something. You can use names of artists that don’t exist and those work just as well, once you’ve figured out what they mean.

Also, even if the model didn’t come with knowledge of an existing artist someone could fine-tune it in, and it’s possible the model can learn about your art style without seeing any of your images anyway.

This is a very important point.

Others have alluded to this, but I'll make a larger claim: human artists actually are doing very little that is original. The styles, composition, and subjects are all derivative works. That means their "essence" can be recreated without them.

Another fascinating consideration, if artists names were excised from future models, either by law or by choice, the "mirror" artists (fake) could be the ones which become famous and the humans forgotten.

AI could become more generalized or there may be millions of models, doing specific things, strung together like with APIs. Either way, if an artist or trademark owner chooses some kind of explicit blocking, from the input side, it is possible that it would be the equivalent of your keyword censored from Google's index, Google Maps, Amazon, and so on. Disney? I don't recognize that word.

It might not be storing all of the images, but it's clearly storing copies of some of the source images.

A few weeks before OpenAI made DALL-E 2 open, I went through a series of prompts using well known artists as a basis. Vermeer was one of them. It generated some pretty amazing works that were 'inspired' but not direct copies of Vermeer's works.

I started feeding the same prompts into StableDiffiusion (the HuggingFace online demo) and it output direct copies of Vermeer's works with minimal modifications.

Do you have the exact prompt that essentially generated Vemeer's works? Omitted in the parent, but I also noted that while it is hard to reconcile the idea of human-directed transformative use with ML models, it might be actually easier to verify the models' plagarism---which is not equivalent to copyright infringement but can be a problem by its own---once you have a right prompt as they are decidable.

"the girl with a pearl earring by Johannes Vermeer" and variations on that, will spit out the named painting.

Two examples: https://imgur.com/a/KAxmZCl

The original: https://en.wikipedia.org/wiki/Girl_with_a_Pearl_Earring#/med...

It's reproducing the folds in the headband, riffles in the clothing, etc. It's not just a "similar" reproduced image, it's clearly some minor variation on the original.

I just typed in 'girl with a pearl earring vermeer' and got this:

https://drop.qoid.us/vermeer.png

And, for a slightly different scenario, here's what I got for 'pikachu':

https://drop.qoid.us/pikachu.png

But I think this sort of thing is the exception rather than the rule (similar to Copilot regurgitating the text of software licenses). Girl with a Pearl Earring is so famous that there are countless copies of it on the internet. I don't know what kind of deduplication process was used for Stable Diffusion, but if you consider multiple photos of the same painting, plus images distorted by compression, or cropped, or embedded in other images, plus attempts to redraw it or even parody it, I can easily imagine that the training set contained thousands of images or more that are essentially the same work. And so the model memorized it, albeit at low resolution.

Similarly, it's seen countless different images of Pikachu.

However, this seems to only apply to the very peak of popularity. I tested some other Pokemon: it does a decent job with Charizard, but with Bulbasaur and Squirtle and Jigglypuff its output only somewhat resembles the character design, and for less popular Pokemon there's no resemblance at all.

> the exception rather than the rule [...] . Girl with a Pearl Earring is so famous that there are countless copies of it on the internet

I think that's part of the point, though.

Even if it's working like some kind of feature detection - "this is what 'nose' looks like [nose1,nose2,...,nose9999]" and "this is what 'girl with a pearl earring vermeer' looks like [vermeer1,vermeer2,...,]"

Clearly it is storing copies of those images internally, and constant re-enforcement of it finding noses and vermeer paintings in the sources pushes the priority of those features up so that it stores them.

We can't trust these models, as they exist right now, to reliably generate new works, rather than spit out something that already exists.

Technically speaking, there can be only so many such reproducible works in the model due to the information-theoretic constraint. But yes, I agree that this is a close enough reproduction of the Vermeer's work, and while this is indeed in public domain, you are right that other copyrighted works may be present in the model, in verbatim and extractable with a right but currently unknown prompt.

The photos of the painting on the internet aren’t original works either. This system is learning from everyone taking pictures. If you paid a very good artist to paint that painting, they’d work from a picture. No artist emerges fully formed without impact of what they’ve seen.

There’s a difference between an artist and a machine (for now), but it’s not unreasonable to assume that some internalisation of others’ work is fair.

Not reliably, no. But at least for this size of model, its capacity to memorize seems to be limited to works that are so popular that 'everyone' knows what they look like. And it also seems to generate them only when asked. So in any situation where a human is reviewing the output, it doesn't seem like there's a big risk of plagiarizing something by accident. Larger models may differ.

I feel like this is one of those occasions where perfect is an enemy of the good. Yes there might be some, very very famous, artworks stored inside it. But unless you think StableAI has completely blown past the current hutter prize winner, there is no way this thing is storing many things. The things it might be storing are too public to really be that big of a deal. It will spit out a Vermeer, but your submission to Divient Art in 2007 is definitely protected.

> there is no way this thing is storing many things.

I didn't claim that it was storing all of the images.

It is, however, quite clearly storing at least some of the original training images. What that definition is, I don't know.

To go back to the original point of this thread:

> The thing is, people have wild misconceptions about what this technology does. Many clearly think it's a photocopier. As if StableDiffusion is hiding copies of all these works inside.

I don't think it's a wild misconception, or even an unreasonable concern to be worried that it could output some of it's training images. As an end-user of the technology, you have no idea.

I feel like you completely missed the point I was making with the perfect vs good argument

It does a pretty good Mario

Another discovery that something thought to be uniquely human really isn't that hard.

It depends on what you mean by "art".

There's different sorts of art for different purposes (commercial, fine, etc.), and for many of these sorts of art having an AI make it would be acceptable.

But when I think "art" without a qualifier, I think "fine art". And the purpose of fine art, for me, is human communication of ephemeral things that aren't possible to communicate in language.

I don't want AI generated fine art because it turns the art into just a pretty picture. It's no longer about another person communicating something significant and human to me. It removes its purpose for existing.

In that view, I totally understand why some art communities would ban AI-generated art. I don't see a problem with it. I'm sure other communities will not ban such art. That's fine, too.

But I know which community I would be more interested in paying attention to.

> It's no longer about another person communicating something significant and human to me.

There is a person thinking about the prompt, typing in the prompt, painstakingly selecting the image that matches what they imagined in their head.

It just less elitist, as now even a child can create a complex, meaningful and beautiful images by leveraging the essence of human art (compressed into a single ML model).

I, for one, never had the time & chance to learn painting. Even if I did it as a hobby, it would take me up to a decade to become proficient at it.

And now I can ask AI to paint what I dreamt about last night (!!!), or create an impressionist painting almost exactly like I see it (or even better).

Fun factor is also there: I can see how an anime character could look in real life, or put a movie character in a new context , or just generate cute corgis wearing funny hats and driving cars

> There is a person thinking about the prompt, typing in the prompt, painstakingly selecting the image that matches what they imagined in their head.

Which does make this a type of art. It just isn't the same form of art as digital painting, the same way painting a tree is not the same type of art as growing and 100 trees, choosing one of them and taking a photograph of it. Photography is definitely an art-form, but it's a different art-form from digital painting, even if the end result of the both of them can be a digital image. Not better or worse, but definitely different.

> There is a person thinking about the prompt, typing in the prompt, painstakingly selecting the image that matches what they imagined in their head.

True, which makes it more akin to photography. I'm not saying it's "not art" (I'm saying the exact opposite of that), but that it's not art that is of interest to me. Much like my interest in art photography is very limited. Different people have different tastes.

> It just less elitist

I disagree. I think this whole subject is orthogonal to the concept of elitism. Sure, there are certainly art snobs who take an elitist perspective, but they're a very small segment of the art world.

How would you describe a Rothko in words? Can you create a Van Gogh painting by incantation?

Can't help but think about the obvious: maybe we should just skip all the drudgery of the natural convulsive progress and ruthless competition, push "accelerate" and get to the mature AGI civilization regime as soon as possible?

And there we should just ask our new creative oracle machines the obvious: "devise an understandable minimally invasive method of uplifting our own cognition and extending our biological lifespan". Surely at least some gene/cell/whatever therapy which does what we need is theoretically possible - the human beings aren't special enough for this not to be the case.

At least that's the zeitgeist I perceive among others who understand the significance of the unfolding progress.

Pushing "accelerate" costs a great deal. Investors will want a return on their capital: either in money pressed from unequal access to the resulting paradise or in direct power over this new god.

With billions upon billions spent on Big Science, asking the government to spend just 1 billion on an AI training run with guaranteed spectacular results doesn't sound too outlandish.

Why should we pay taxes, if the government won't even train AI for us, given such opportunity?

AGI is an existential threat to human-led governments. Why would they build something to displace them?

We all know that analogies from Hollywood classic scifi movies aren't productive or even at all useful in our mundane reality. The general-purpose AI is going to be a very useful, obedient, capable tool, and the great powers are going to seek ownership over it, like they do with various military and space technologies.

Current safety techniques are far from sufficient for accelerating AGI progress to be a good thing.

AI safety discourse is mostly still in 2000s, with outdated intuitions informing loads of rigid concepts specific to this time and community.

Large language models trained on cross-entropy loss for next token prediction aren't going to be dangerous per se, even in the limit - simply due to the nature of the objective and the distribution they are set to approximate.

We should make sure these powerful tools don't fall into a small exclusive set of power-hungry interests, though. Transformative tools such as these should benefit all of humanity and let it prosper.

If large language models trained on cross entropy loss for the next token prediction in the limit aren’t dangerous, then they aren’t enough by themselves for AGI.

An AGI, by virtue of being at least as general as a human, would be able to effectively form a plan for how to act in the world to achieve fairly general goals, in fairly general conditions.

Even if it isn’t agentic, if a program can formulate these plans in a way people can use, then the combined program+user still implements the optimization process.

That's why we have to make sure the user (committee) is safe and aligned. The usual silicon valley tech execs, or east coast financial bros aren't going to cut it this time.

The democratic solution here looks better than the expected rule of the (very) few.

That isn’t enough if the plans which are enacted have unintended undesirable consequences.

Yeah, give me a second, singularity exists.

It seems to me that the human effort spent in creating the original images in an AI's data set is an important element to consider when thinking of the "difficulty" here.

Certainly, but those human artists were also trained on a data set their whole lives. Raise a baby with no training data set and they might produce a handprint on a wall or a rectangle of facepaint. There's a reason indigenous tribesmen don't spontaneously invent hyperrealism.

However, there's a major misconception about training of humans and AI.

Image generating AIs are trained with massive amounts of images and text, however image generating humans train with much broader spectrum of experience.

Also, feeding a model tons of images created by humans (directly or by proxy) and claiming that AI is generating something completely new is a bit naive IMHO. Humans mix a much more broader and deeper experience pool to create things without prompts.

An AI model blurts out something derived from a corpus of images and text created by humans, that's all.

The technology is impressive for sure, and it marks a new era in terms of possibilities, but it doesn't take my breath away, sorry.

Further to your point, comparing humans to AI is not just misunderstanding AI, but also looks past how humans create and choose the styles which this AI is now reproducing through a statical approach. Without the human guidance the AI would be bland, AI is limited to the walls of whatever it has been exposed to - humans are not.

The AI emulates. Humans create. - The significance here is not trivial.

Humans emulate as well, and produce bland art as well.

I've seen lots of human-made art (especially on Artstation) that failed to elicit any emotional response in me.

The reason people are having this discussion is a fear of AI creating aesthetically pleasing and deep emotionally through statistical approach.

And the deeper fear that AI can potentially "hack" our senses by providing us with exactly what we want to see, read or hear (statistically).

i.e. if some subset of art is pushing our biological buttons more than all other art, and the effect is generalizable over human population, what would that say about us?

I think the point has been missed. AI is bound. Humans are not.

To better illustrate that difference: Just because anecdotally some human might produce crap doesn't mean that is the limit of all humans.

Well put. AI is an evolutionary death, in a sense. True Intelligence is not, at the point at which it is not, it is no longer artificial, and then ethical concerns must apply (rights for so-called AI citizens, etc.)

>AI is limited to the walls of whatever it has been exposed to - humans are not

So you can imagine new colors?

Humans mix a much more broader and deeper experience pool to create things without prompts.

That wide variety of experience changes the decision of what to create, but I think it has less influence on the actual output than one might expect. A person can have enormous life experience but if they have never seen a logo, a photo, a painting, a drawing, or any other kind of rendered image they will be no more sophisticated in their artistic output than a caveman. Modern artists stand atop a mountain that their ancestors had to climb inch by inch.

Actually, I don't agree. I take photographs and design logos for my pet projects sometimes, and I optimize for feeling most of the time.

I want my photos to make the viewer feel a particular way when viewing what I took. This affects colors, style, and everything.

Arguably, as long as the medium carries the message the creator intended, it's equal in my view. It can be a photo, a drawing, a digital painting or a physical painting with oils. They can create the same feeling if the artist aims for it.

This "optimization for feelings" is a result of my own non-photographic experiences. Sometimes a song, sometimes an event I went through, sometimes an art piece I saw on a medium I don't work.

From my understanding and observation, I saw that many artists work that way. They reflect their emotions in one domain (generally personal life) to another domain (the art they create). Also, there was a video which I fail to find over and over which shows how three designers got affected by the things they saw (and deliberately planted into their minds slowly) during a 20 minute car trip.

Yes, we're building upon this great mountain of experience and knowledge, yet our output is affected by what we experience in other parts of life, unrelated to the art we create.

Consider the following experiment: The music you listen changes your mood, your working speed, and what you feel; effectively changing how you operate and what you create. Exclude work from this, which is by definition something you have to do, unless you're literally dying.

"yet our output is affected by what we experience in other parts of life, unrelated to the art we create."

Right, and what happens when you let that loop feedback to itself?

An asphalt road on every corner, so we build more asphalt roads, we design only what has been designed before. Amplify this thousands of time through computer generated design, the feedback loop is closed and unchanging. Unless your goal is actually to create an independent organism (different moral issue), you are creating sophisiticated feedback loops, not worthwhile content generation. Unless by content generation you mean endless remixing...which is not the same.

Sort of like changing the color of a 3d model and claiming it is a new race with new attributes, as is often done on the cheap e.g. in videogames.

> claiming that AI is generating something completely new is a bit naive IMHO.

I don't see how AI art is less new than the vast majority of human art. Both can create unique compositions that are still deeply rooted in patterns and principles thousands of years old.

> Humans mix a much more broader and deeper experience pool to create things without prompts.

AI does not have understanding, but it's stealing the underlying patterns that are the end result of that human experience.

Absolutely. Though I think that speaks to one of the ways we tend to discount the difficulty of developing human artistry, rather than a defense of AI gen being "not hard".

Yes, that 4 GB of data can't be really independently generated, but the fact that a large chunk of that effort can be compressed into 4 GB is still amazing.

If your life's work can be compressed down to 2 bytes, it wasn't that hard.

OTOH, if the 2 bytes is an index into a larger corpus of collected works, the question becomes more interesting.

OTOH, if the corpus is an ensemble of other 2-byte objects...

Who would've thought art would get automated before driving.

That's like saying "well, making knives is easy" while ignoring everything that is needed to make materials then shaping them into final form.

It's only "easy" because bunch of people over years perfected every step of the process.

It doesn't compress each image separately. Rather, it compresses its entire training set into one dense region of cartesian space. Each pixel of each training image is in there, scurnched up next to every other pixel of every other training image. It's like sardines in a can, except in thousands (?) of dimensions. You have no hope of calculating how much space it needs for that just by dividing the size of the model by the dimensions of images in pixels, like you did.

You can see that it has memorised all its training material very clearly in its ability to perform style transfer. You can ask it to geneate an image of Pikachu in the style of Vermeer. It can do that because it has memorised images of Pikachu and images by Vermeer in its weird compressed form and it can combine them by finding a point on a gradient between their representation in its memory.

If it didn't have memorised images of Pikachu, or images by Vermeer, how do you guess it could generate images of Pikachu or in the style of Vermeer? It's not magick.

> ... sardines in a can ...

Better analogy: suppose you take 10 pieces of A4 paper and scrunch them up into 10 separate balls. Now suppose you scrunch them all up together into one ball. One manifold, taking ten times less space.

In the same way, you don't need to store each pixel of each image separately, or in its own sub-space. Yes? It's really good compression, that's all.

20 bytes? Wouldn't that be about 2 bytes per image?

Ah, yes, a sloppy calculation in my part. I probably wanted to say something like 20 bits in the original tweet.

I've used the words "stock photography" in an image and saw the shutterstock logo across the image in the output. Not sure how that happens but try it out.

Textual inversion also takes an entire concept (including context, style, and 3D attributes) and fits it into 768 float32s / 4KB.

That's a different topic, as you need the entire model to "decompress" that 4 KB data. I'm talking about the model itself.

Reply on news.ycombinator.com