Hacker News Reader

> If I've "learned" Harry Potter to the level where I can reproduce it verbatim, the reproduction would be a copyright violation. If I can paraphrase it, ditto.

Yeah, that's something that I've not seen a good answer to from the "everything AI does is legal" people. Even if the training is completely legal, how do you verify that the generated output is not illegally similar to a copyrighted work that was ingested? Humans get in legal trouble if they produce a work that's too similar. Does AI not? If AI doesn't, can I just write an AI whose job is to reproduce copyrighted content and now I have a loophole to reproduce copyrighted content?

If gen-AI had been used to produce Nosferatu there still would have been a case, right?

Seems like so much tech "innovation" these days is really just to sneak around laws and social norms in pursuit of a rent-seeking position.

Nailed it. All of the “progress” in the last, say, 20 years is exactly this. They call it “disruption” and wear the title “disruptor” as a badge of honour.

> Humans get in legal trouble if they [???] a work that's too similar

If they sell a work that's too similar.

What intellectuals do is quoting. Of course legal.

I think you have to be practical. It would be difficult to train an AI to consume Harry Potter and compress it but prevent it from recreating it. You can try and people do, but there are always ways around it.

But it's on an individual prompt basis. It's not like ChatGPT can produce the entirety of its text and sell it as a pdf. It's just a device that could reproduce it much like a word processor is a device that you can read the book and type out the contents.

So the question is one of practicality. Do we ensure that no copyrighted material is in the training data? Difficult but probably not impossible. But what you can't do is target the content in all its various other forms, from descriptions of the plot, reviews, fan fiction, etc. So in the end its pretty much a lost cause.

So what to do about it? I don't know. In the utilitarian sense, I think the world in which this technology exists in a non-crippled form is a better richer world than one in which there are all these procedural steps to try to prevent this (and ultimately failing).

Whats the harm here? Are people not buying Harry Potter books and just having an LLM painfully recreate the plot? I would imagine Harry Potter fans would be able to explore their love of the media through LLMs and that would drive more revenue to Harry Potter media, much like fan fiction and pirated music lead to more engagement and concert sales.

In the case of new art, maybe fewer artists get commissioned, but let's be real, Mike Tyson wasn't going to contract out an artist to create a ghibli style animation of him anyway, so there's really little harm in LLMs here to artists. If anything it expands the market and interest.

> So what to do about it

We proceed towards AGIs that implement proper understanding, and have them read all of the masterpieces and essays and textbooks - otherwise they will be useless -, as is fully legitimate in any system that foresees libraries.

I'm just going to briefly respond to the part you wrote about art in particular.

We may not have a way to actually quantify the harm that GenAI is doing to creative industries because some of the damage is long-term. Choices are being made right now based on the state of the world. Why would anyone start an art career in this climate? What does art as a profession look like in 5 years? 15 years?

Art is not just the final artifact, and I feel we're surrendering part of our humanity in service of enriching big tech companies.

Cleanroom implementation comes to mind.

If I just remember the source code of a 100 line program and then reproduce it verbatim a week later that doesn’t suddenly make it a new work.

This is why I don't believe in restrictive licensing of open work.

>Even if we accept the house-of-cards of shaky arguments this essay is built on, even just for the sake of argument, where Open AI breaks my copyright is by having a computer "memorize" my work. That's a form of copy.

No, it isn't, unless you're also going to call it copyright infringement when search engines store internal, undistributed copies of websites for purposes of helping answer your search queries.

Edit: or, for that matter, accessing a website at all, which creates a copy on your computer.

Correct, the above posted ignored the four prongs of fair use:

1. Is the work more educational or commercial.

2. What is the nature of the underlying work (creative works are more protected).

3. Is the work transformative.

4. What will be the impact on the underlying work's market.

Search engines do not make their internal copies available, compete in an entirely different market (that benefits the makers of the underlying works) and are considered quite transformative because they enable discovering vast information on the internet.

On the other hand, almost zero LLMs/Text-to-Image generators are educational in nature (certainly none of the ones being sued for copyright infringement). They frequently are trained on highly creative works like art and writing. Some of the work could be transformative, depending on where the learned data manifold the output of your request lies on, but a huge amount is similar to the training data. Lastly, these models have an outsized negative impact on the underlying markets, by being vastly cheaper compared to a human's labor, and for dubious quality at that.

What is that replying to? I was addressing the argument that "well, you stored the work in memory, therefore you infringed copyright". It seems like you agree that that jump doesn't follow, just as I argued?

You are totally correct, I misread your original comment and thought you took the opposite stance.

You're making a classical error programmers make when encountering legal systems: Treating law with the same rules of logic as code.

I can cite case law carving out lines around things like copy in RAM/SDD/etc. accessing a web site (very clearly legal) or internal copies for search engines (very close to the border, but still legal), but courts -- at least in systems based on common law -- don't work like a computer program or a mathematical argument.

So the general argument "X is like Y" doesn't work if you're looking purely at the technical action. The exact same action at the level of bits and bytes can be legal in one context and illegal in another.

What matters to courts are things which are less easy to formalize: intent, impact, etc.

Just as with computer science, there is a rigorous and formal analysis, but very different from the one you're making, and operating under very different rules.

That's an unfair reading of my comment.

For one thing, I was replying to your own attempt to do a programmer style legalism, of "welp, you instantiated a copy, therefore it must be copyright infringement, case closed". I was already basing that on the principle you're now appealing to, to say that that isolated fact alone doesn't settle the matter (which is a much lower bar than the broader point that, all things considered, generative AI isn't infringement).

It's great that you recognize that programmer-style legalisms don't work in the law! It just wish you'd had that insight in mind when making your original comment.

Second, to the extent that I endorsed the general argument (generative AI not infringing), I wasn't claiming that the point worked because of some mechanistic technicality, but because of the very broad trend of accepting internal copies for a wide range of saleable data products. That is, in fact, how analogical reasoning works in law: courts compare the case at hand to similar ones.

Is any one point definitive? Of course not. But brushing off all analogies is unjustified deflection. If you think there are more pressing considerations that override the pattern I've pointed to -- like why the "intent, impact, etc" matter here -- great! I'm happy to have that conversation. But you can't just gesture at "legal rigor is hard" as if that's some kind of counterargument.

> If I've "learned" Harry Potter to the level where I can reproduce it verbatim, the reproduction would be a copyright violation

Of course not: in fact, memorizing has always been a right. (Edit: that would depend on what is meant with "reproduction" though. As written elsewhere, properly done quoting is of course fair use.)

> If I can paraphrase it, ditto

Even more so: people have lost legal actions because they sued authors of parodies.

Afterthought: I realize my reading of the original post should have been more attentive;

nonetheless, I would still object that the ideas proposed (e.g. "reproduction" and "paraphrasing") cannot be taken without further specification. The fault is not at that generic level. And the context we should be after should not be this provisional one, of an unfinished technology.

Maybe the infringement occurs when a user uses the model to produce the facsimile output.

Good idea. Let’s make it a minefield of copyright infringement for the user so they never know whether it’s emitting something novel or it’s emitting AGPL code.

Why the heavy handed internet sarcasm? Youtube has handled this exact issue with fingerprints for a while.

That's what the legal department at my employer (huge multinational corp) came up with: When an employee uses one of the approved gen-ai models it's on the employee to check the output isn't infringing - legal argues that not doing so would be grossly negligent, making the employee personally liable.

If my company had a policy like that I would just not use AI tools at all for anything.

Best possible outcome. Unlikely to be, but maybe even the intended outcome from legal's POV.

I agree that you should call the output copyright infringement (or not) without regard to how you got there. So, if it produces an identical copy of the text, or of e.g. Indiana Jones, that you then distribute it, sure, that is copyright infringement.

But the mere act of using them for training and producing new works shouldn't be! In fact, until 2022, pretty much no one regarded it as a copyright violation to "learn from copyrighted works to create new ones" -- just the opposite! That's how it's supposed to work!

Only when hated corporations did it with bots, did the internet hive mind suddenly decide that's stealing, and take this expansive view of IP rights (while, of course, having historically screamed bloody murder about any attempts to fight piracy).

Scale matters. There's a difference between individual humans taking time to study copyrighted works, and so-called "AI" doing it on the scale of the equivalent of millions of man hours.

But what about that kind of scale matters to this particular area? The article spends a lot of time showing the substantive parallels between how humans and AIs learn e.g. in how they never store an exact copy, but a high-level understanding that rarely produces anything verbatim.

> If I've "learned" Harry Potter to the level where I can reproduce it verbatim, the reproduction would be a copyright violation.

Yes.

> If I can paraphrase it, ditto.

Not necessarily. Summarizing, for instance, is typically free use.

How about not simply summarising, but creating a broadly similar novel, with similar characters which undergo broadly similar story arcs?

I can see how it could end up as infringement, but also how one could avoid infringement. The issue would seem to be if the original author wishes to sue.

As an example, compare Brooks "The Sword of Shannara" with Tolkein's "Lord of the Rings". It is widely accepted that the former is heavily derivative for the latter; in my view essentially being an example of the above broadly similar novel.

Yet AFAIK, Tolkein's estate didn't sue Brooks. However if they had, how likely would a victory have been?

Now in the case of "Generative LLM", we may have an even closer "derivative work". However the LLM users could well get away with it if they consume a wide enough canon, and the source authors either do not learn of the derivation, or do not have the means to sue.

One of the key lessons learned in the past half-century is that lawsuits like that are:

(a) Often won

(b) Pyrrhic victories

Right now, it's the policy of Paramount not to sue over very clear violations of Star Trek copyrights (see: Star Trek Continues), Rowling to not do the same for Harry Potter (see: Methods of Rationality), etc.

A lawsuit like that results in a modest payout and take-down, together with a loss of brand worth many times that.

On the other hand, the derivative works tend to drive brand value, recognition, and sales.

That's why many works have clear guidelines supporting and encouraging derivative works, with specific boundaries to make sure they don't undermine those works.

I don't know The Sword of Shannara, but each time someone says "inspired by Tolkein" or "building on themes from," Tolkein increases in brand value as a classic and then THE classic. There is a legal line as to where inspiration becomes copying, but if we were to assume, arguendo, that it's clearly over that line, it still probably wouldn't make sense to sue. At best, that would have a chilling impact on more works in the "inspired by" category.

> If I've "learned" Harry Potter to the level where I can reproduce it verbatim, the reproduction would be a copyright violation.

By your logic anyone with good enough memory violates copyright law just by act of remembering something.

No - it's the reproduction, not the memorization.

It's reproduced in the memorizer's neural connectome.

So training AI on copyrighted data isn’t a problem unless it spits out the data verbatim. Correct?

Close.

It isn't a problem until it spits out a similar enough copy.

Copyright violation doesn't have to be verbatim; taking a 4k movie and reencoding it to 320x200 before distribution isn't legal.

Memory? No.

Reproduction? Yes.

If you write down a sizeable quote, that can be a copyright violation.

This doesn’t seem true. I mean, it might be true if memory could be seen or manipulated, but what would you bring into a court of law to prove that I remembered something too clearly?

I don't see that point in the original comment. Remembering copyrighted content ≠ reproducing a verbatim of it.

No, because you don’t actually violate copyright law until you produce and distribute copies.

It’s perfectly legal to memorize a book, type up a copy from memory, and to even print out a copy that you keep for yourself. But as soon as you start trying to sell or distribute copies, even for free, now you’re breaking the law as written.

That was actually the case in music until a decade ago or so. Led to ridiculous lawsuits (for example Ed Sheeran’s).

Previously, artists needed to prove they hadn’t heard the song that they were accused of infringing. That was virtually possible because there’s a lot of music you could hear anywhere, even just a car driving by. Artists continuously lost these court cases.

Nowadays the burden of proof is luckily no longer on the defendant. But I think that only changed a decade ago or so, thanks to some efforts by music industry lawyer Damien Riehl. I know, ridiculous.

What's stopping me from paraphrasing movies by peppering the least significant color bits? Would that make copying them legal?

By that reasoning any VHS copy would be legal. Pretty sure Hollywood takes a slightly broader view on what constitutes a copy

I'm sure that Hollywood would charge me for randomly remembering scenes if they could. What makes them the ones to dictate the rule?

Even if we define “AI” as lossy storage with random error insertions, it still amounts to unlicensed reproduction.

Like saving an image as a JPEG doesn't make it a new work

> if we define “AI” as

something completely oblivious of the past, it would be a great disservice.

> as lossy storage

well that would just be a great misunderstanding of what "learning" is.

> well that would just be a great misunderstanding of what "learning" is.

You need to assume a malicious actor here, its trivial to use an AI as lossy storage, if that is legal copyright washing then large corporations will do so at industrial scale.

> its trivial to use an

You mean, "a Neural Network as function approximator for the reproduction of texts". But in fact, AI cannot be confused with that idea - and

be it AI ("system for the replacement of a professional"),

or AGI ("system implementing intelligence"),

or LLM (which can only be useful as "system with emergent properties from text generation"),

"learning" is strongly opposite to "memorizing" (as would be evident in a student - trivial example, one that would memorize a multiplication but be unable to perform it).

So, that somebody places a sticker on his apparel to fake a profession for «malicious act[ion]» cannot make us forget the real thing: "learning" is what is sought, and for learning we need the teaching material.

...Which, societally, is rightly decided as free (i.e. stored in libraries).

Exactly- the information wasn't created from whole cloth, but rather by reading (copying inside a computer from disk to ram to gpu) the copyrighted information in the first place.