Hacker News Reader

Comment by usrbinbash | Hacker News Reader

> scale commonly matters in law

I am not a lawyer, so the following is only my opinion.

Scale matters, but so does the legality of the thing that scales.

Reading two dozen books by other authors, or studying hundreds of artworks, or visiting the museum of awesome statues every week, in order to get inspired for ones own novel/painting/scuplture, isn't illegal.

So a lawsuit will have a really hard time argueing that it somehow is a problem if its two dozen billion books/paintings/sculptures. Because, such a lawsuit would suddely need to explain why the smaller scale is also problematic, only less so. And given that this is basically how art worked ever since the first human had the idea to paint pictures on a cave wall, that's a hard sell.

View on HN

However, AI learning is not the same as a person learning. The same way memorizing a book is not the same way as putting it into computer memory. Nobody would sue you for copyright infringement if you memorized a book, song or movie in your head. No the issue is a completely different matter if you made a copy on a harddrive.

It is superior, but it is absolutely still learning.

ML can do either facsimile or imitation far better than a human mind can.

You seem to be conflating both things and suggesting that ML only does facsimile, which is where the potential legal problems are.

It is superior, but it is absolutely still learning

No, it is not. Memorization =! understanding.

I can teach a parrot to spew the times table, good luck getting it to understand how to apply it.

And a parrot is billions upon billions of times more capable than any current AI algos.

You don't need to understand something to go beyond memorization and copies.

You don't need to "understand" art to create it though. Good art, sure, maybe, but not really. Plenty of brilliant musicians who know fuck all about music theory, but they can crank out tunes.

Diffusion would appear to me to work in much the same way. It doesn't understand what's good ("works", creates acceptable output) or why, but it knows it when it sees it, and has the tools to refine it.

You’re talking about facsimile, which is a product of memorization, which is a type of learning. And it is not the only type of learning AI is capable of.

AI, like humans, is capable of both imitation and facsimile. It is far superior at both feats.

Your fallacy is that you are noticing AI is superior at facsimile and erroneously assuming it is “not learning”. You are also ignoring the other amazing learning feats of imitation in front of you.

Parrots are lovely animals, but it’s unclear what you think you’ve accomplished by bringing them up. The fact that they are capable of more than just memorization does not differentiate them from advanced AI models, which are rapidly gaining all sorts of abilities.

Polly the parrot would have a hard time producing a picture of Elmo with a light saber in a Superman costume riding a dragon on the moon in the style of Rembrandt (in under 300ms, at least). I also know a parrot couldn’t write a 500 word story about the image.

> And a parrot is billions upon billions of times more capable than any current AI algos.

I am pretty sure that GPT-3 is a lot more capable than a parrot in transpiling a function written in Python to Golang, or writing a summary to a tech-magazine article.

Same as StableDiffusion is ALOT more capable than me in drawing, painting and generally making up pretty pictures.

> The issue is a completely different matter if you made a copy on a harddrive.

But there are no copies. For example, the LAION-2b training data is a total of 240 TB. The pruned SD model based on this dataset, is less than 5GB.

The data isn't copied into the models, it is used to teach the models, letting them learn patterns in the dataset.

You are just anthropomorphizing the model by calling it teaching and then implicitly equating that it's the same thing happening in the human mind. That's your burden to establish when you say it's the same.

Actually, isn't the burden of the plaintiffs to prove their copyrights are being violated?

Whether or not it's identical to human brains isn't the matter, they'd need to prove how a small 5GB model trained from a huge dataset infringes their rights specifically.

Well if it's their defense they need to substantiate it. Obviously the plaintiff has a theory, they filed the case.

I am not anthropomorphizing anything, because this is literally what happens. The model is taught, by having its predictions tested against examples, how images work.

You are, that's not what teaching means and it's not what learning means.

Okay, then what do these 2 terms mean?

Okay, but can you prove that?

I have a 1.9 GB mp4 file on my harddrive. It contains 2 hours and 15 minutes of 1080p video data at 24 fps. Assuming it was generated from 4096x2160 16-bit color depth source material, the "training data" was 10.32 TB. I bet I could even get a similar size reduction as LAION-2b if I recompressed it to 720p.

Could I not also claim that I created an advanced AI model, which did not copy but learned patterns in the dataset? Modern video compression algorithms are getting quite complicated, after all.

I think no reasonable person would agree with this, but can you prove that the AI model is doing something substantially different?

> Could I not also claim that I created an advanced AI model, which did not copy but learned patterns in the dataset?

Such patterns would enable the video file to decode into a multitude of pictures not originally in the training data. Obviously, a video file cannot do that...it's just compressed data.

Generative models however can generate things that are not in its training set.

And of course, there is a fundamental difference in the source data between compressed video and a generative model: video codecs work with a sorted sequence of images, where most images are slight variations of the ones before them. The training for generative AI doesn't have these properties, the input is not an ordered sequence, and even similar pictures are not sequential variations of one another.

To expand a bit on this:

Relying solely "uncompressed" size does not a really good metric make (this is analogous to the raw input size of the LAION dataset): one could make a reasonable argument that there are not billions (1) of image-pairs that are effectively identical up to a minute shift. I would posit the correct basis would be the Shannon entropy of the "best fit" ordering (minimize inter-frame diff), versus the lossy-compressed video, and a similar "best fit" ordering for the LAION dataset vs. the model.

My suspicion is that one will find that the relative number of "smooth transition" pairs in LAION viz the whole will be very different from the video.

-------

(1) - Napkin math: There are about 194400 frames, so ~37 billion (37791165600) frame-pairs. Assuming you have runs of about 1 second between hard cuts throughout, so an incidence rate of 1/24 for non-smooth transitions, gives us about ~36 billion "smooth transition" frame-pairs. I think it is safe to assume "on the order of" 1 billion, then. This ignores long "action" scenes with significant variance in images throughout, but also ignores longer-than-1-second slower scenes, hence the order-of-magnitude shrink in the assumption as buffer.

> Nobody would sue you for copyright infringement if you memorized a book, song or movie in your head.

No, but what if you then produce "your own" rendition, or "remix", of that book, song or movie and offer it to the public? E.g. you memorize a collection of Taylor Swift's latest songs, and then start performing a medley of her hits in your local clubs, you may well find yourself in trouble.

in order to get inspired

But that never, ever, ever happens with current AI. it is not AGI. it is inspired by nothing, has no creativity, nada, ziltch.

You make it sound like we should assume that getting inspiration from a few hundred or thousand art works that are very famous and highly public is the same as training over nearly every available public piece of art. I see no reason why that should be our null hypothesis.

Humans either learn art by being natural art geniuses, or by receiving instruction and learning through an iterative process (where, again, they might create thousands of art works, but nowhere near the scale here), which is very different.

> which is very different.

Why is it different?

The only difference that matters, is scale. And again, if I want to argue that something done 10000000000 times is legally problematic, I have to be prepared to explain why doing it 10 times is problematic as well, only less so.

Nothing new. People differentiate between genocide and murder, for example, or poisoning water supply vs an individual poisoning. Criminal law in quite a few places definitely has scale considerations.

You just gave two examples of where both ends of the scale are illegal, which only strengthens the argument of GP. IANAL, and I'm not stating anything about the reality of the judicial system, but only following the logic of the argument.

My examples only were meant illustrate that scale is a well known "thing" in legal systems and I happened to pick things with two illegal endpoints (IANAL). You could look at other things like the need for permits for certain things as a function of size and use, if you want simple examples for scale mattering and legal endpoint(s).

I am no lawyer, so the following is only my opinion.

I am completely aware that scale is a "thing" in legal systems. But as I said before: For scale to be important, the unscaled act in itself has to be problematic already.

IANAL, but not sure I necessarily agree I with that. Why then worry about monopolies or cartels? Price setting is just "at scale" there, for example.

Granted, things around power concentration have deep philosophical and social roots.

Would you give me a second of your time? What about 1 billion seconds of your time?

Could I have a dollar? What about a billion dollars?

The burden isn't on me to explain why something being scaled up by a billion is not the same.

The question isn't if a scaled up thing is the same, the question is if a legal thing scaled up suddenly becomes illegal for no other reason than being scaled up.

1. An AI has a training set of one image and produces an exact replica.

2. An AI has a training set of every image in the world and produces an entirely unique work.

Which do you have more of a problem with?

#2 is pretty interesting. After all, the purpose of copyright law is to encourage creative works. If we have machines that can generate creative works on demand with little effort, what purpose does copyright law serve?

I am not a lawyer, so the following is only my opinion.

Copyright law serves the purpose of peoples works being protected from unauthorized parties making copies of their works, and profiting off them.

It doesn't protect from technology making the production of new works cheaper, faster, more efficient. An artist using photoshop can be, and is allowed to be, many times faster than one using oil and canvas.

That's a false dichotomy.

I don't mean to suggest those are the only options, or that either one is even practical. The point is to determine where the objection lies: with the method or the outcome.

Hm, okay. In that case, I would say the first is a problem, and the second is also, but differently.

I think the objectionable thing about the second is that the AI knows everyone's styles, and so can use them in creating something new. Even if the AI is restricted to not be able to paint an image in a certain artist style (as the new version of stable diffusion is, for instance) and the art is unique, I think part of the problem is that the AI is still (presumably) leaning on the collective styles of everyone it has trained over.

If we can train an AI over a small dataset or maybe even a large dataset of old art, or some mix in between, and then maybe fine tune it wtih a snall sampling of modern art, then I believe it would be unobjectionable, as this is largely how humans do it.

Reply on news.ycombinator.com