Are ChatGPT and Other Similar Systems the Modern Lernaean Hydras of AI?
arxiv.org
arxiv.org
Are Altman And Yudkowsky the Troilus and Cressida of AI?
Is AMD The Agamemnon Of Machine Learning?
(a) immediate changes to the licenses for open-source code created by developers that will allow access and/or use of any open-source code to humans only;
(b) we suggest revisions to the Massachusetts Institute of Technology (``MIT'') license so that AI systems procure appropriate licenses from open-source code developers, which we believe will harmonize standards and build social consensus for the benefit of all of humanity rather than profit-driven centers of innovation;
(c) We call for urgent legislative action to protect the future of AI systems while also promoting innovation; and
(d) we propose that there is a shift in the burden of proof to AI systems in obfuscation cases..."
That would be the most hilarious outcome possible. Oops, you can't store open source code on your computer, the access is only for humans!
But also, non-human actors cannot create copyrighted works, transformative or otherwise. For example, a photographer once gave his camera to a monkey and it took some pretty interesting pictures. He tried to get copyright for these photos, but the Copyright Office ruled that he wasn't the one who created them and the monkey is not a person, so the photos are not copyrightable at all. The law could be changed in these cases, but if we're treating the AI as a creative agent rather than a reservoir of other people's works that humans can request from it, then it seems to me like AI-created works should simply not be copyrightable.
A derivative work is something like making a collage or a video game modification/sequel.
An AI isn't really either of those. It doesn't contain the original artwork in any sense, and as a trained numerical model it's not a derivative of any other works it was trained on. No more than you looking at the artwork and learning aspects of it at least.
In the copyright sense it should be treated like you treat a person. If it produces something that is clearly a derivative copy of something. Hit it for copyright. Otherwise, treat the output as novel output.
If you use an AI you need to be held liable for whatever you take from it and distribute, if you violate copyright you violate copyright.
But the AI itself is no such thing. It's a model trained on billions of examples which fundamentally cannot contain all of the works it was trained on because it's just too small of a model and too large of a data.
To sue over that containing artwork would be like to sue over Photoshop containing artwork because you can produce a derivative work based on it.
Except that Photoshop will not regurgitate someone else's program practically line-by-line when you type the first 7 words of it into a prompt, which Copilot definitely does.
> In the copyright sense it should be treated like you treat a person.
A person with a camera and/or a copy of the original code, if anything. The fact that it compresses very well doesn't remove that these systems have almost token-perfect memory capable of recalling it on demand.
> as a trained numerical model it's not a derivative of any other works it was trained on. No more than you looking at the artwork and learning aspects of it at least.
It is definitely a derived work of the programs it was trained on, which is kind of obvious since it can produce segments of them on demand.
This is no different that compressing my program and then trying to claim the compressed file is no longer derivative of my program since it "just cannot contain my program anymore".
Whether (as a person) _by using a model which is derived from my programs_ you are immediately creating a derivative work of my programs is a different story, but I think it is rather safe to assume that if you just copy paste large swaths of the output of the model (or even simply look at them), your program is also going to be derivative.
You could argue that the people who have that specific bit of code would have a decent right to sue, because they have real definitive proof that their works are in that model in some form.
But unless you have managed to make such an output for (insert work here), You have no real claim to copyright on that model.
Which is why I made that Photoshop analogy. You can use it and get it to output someone's copyrighted work. The fact that you can do that doesn't make photoshop itself a problem. The act of making it spit out that copyrighted work, then distributing it, is the problem.
The liability for that is going to have to be figured out in the court system.
> A person with a camera and/or a copy of the original code, if anything.
No, the model doesn't have a copy of the original code when it's running. It only has what it was trained on, as if you read a bunch of stuff on GitHub and later on use what you saw in there while you were writing your own stuff.
And people totally do this as well, they will see things and learn from them and accidentally or unintentionally reproduce them when they are working later.
Which is another reason I think you should treat the output of these models as if someone has actually done it themselves. If you can prove there was violation there, there was violation there. Otherwise, you can't claim the model just like you can't claim the person who learned from your stuff.
> The fact that it compresses very well
It doesn't. The data of these things are trained on is absolutely massive and the final files are not nearly so large. The compression ratio would be absolutely bonkers if that was what was going on here.
What you're seeing is the effects of overfitting on specific examples, which doesn't represent the whole at all. This is finding an exception to the rule and taking it out of context.
To say that the model itself is a derivative work is absurd. You've got a picture and you're going to claim that this AI model that was trained on 7 billion pictures or something like that is derivative of your work?
> This is no different that compressing my program
It is very very different than compressing your program. A compressed file is a compressed file, builds to be compressed and then decompressed to get the original file back.
Again, except for the example where you own a work that is being reproduced by one of these models, proven, this is more like claiming a hash of your file in a folder with 100,000 other files is a derivative work of your file.
Note that it is _my_ code that Copilot regurgitates, and at least 50 lines of it as I kept prodding it. I didn't actually put it in Github, but some scammer decided to mirror it there. It is a popular program, but not that popular, so I just can't accept the argument that "these models reproducing a copyrighted program is a rare situation".
It is actually a frequent occurrence, and this dismantles the majority of this type of arguments.
> Which is why I made that Photoshop analogy. You can use it and get it to output someone's copyrighted work. The fact that you can do that doesn't make photoshop itself a problem.
Photoshop analogy is still not really working. These models have been trained on copyrighted programs and can reproduce portions of them on demand based on short prompts (or at least significantly shorter than the actual portion of the program). Photoshop _cannot_, unless you literally load the original artwork on it or literally reproduce it in the program action by action (or pixel by pixel).
There is simply no way to argue that Photoshop is a derivative work of 3rd party X's artwork, specially if Photoshop creators didn't even have access to X's artwork; but there is ample opportunity that the model _is_ a derivative work of 3rd party programs, specially when by their own admission they used my program's code to build their tool.
In addition, and based on a strict interpretation of today's legal definitions, I could build a case that model infringes on my copyrights, from the mere fact that my programs were used for the construction of the model. It doesn't really matter what you think about the particularities of the model; there was no human in the loop, and therefore it is impossible for the model to be anything other than a derivative work. For Photoshop I just cannot have a case (unless, e.g. the brushes of photoshop were "trained" by a program from my artwork).
> The data of these things are trained on is absolutely massive and the final files are not nearly so large. The compression ratio would be absolutely bonkers if that was what was going on here
Yet it happens. Is it a matter of numbers? Would a very clever compression algorithm suddenly strip from its output the copyright protections of its input? Unless it was a bonkers algorithm, like always decompressing to the same string, I rather doubt anyone would claim that.
> You've got a picture and you're going to claim that this AI model that was trained on 7 billion pictures or something like that is derivative of your work?
The model is literally trained on my picture; how can it _not_ be a derivative of my work?
Not to mention that the discussion quickly goes moot when the model can reproduce _my picture_ (in this case, my code) on demand, as is the case here.
If it could reproduce my code without having been trained on it, you could have had a (weak) case... something equivalent to clean-room reverse engineering. But this is not the case at all; these models have literally consumed copyrighted code as input.
I think you're doing here is connecting two very disconnected things.
I agree with you entirely that cases were a model is overfitting are copyright violations. I think you should have the right to sue open AI or whoever it is that made the model over the violation of distributing what is basically a copy of your code when you can go in there and reproduce it so exactly.
I do not agree is the idea that you think that this means that training on your data is a copyright violation.
If the model was trained properly and set up so that it didn't overfit and it didn't reproduce your exact code, you would have absolutely zero right to any claim of copyright.
The fact that your data was trained on is not the problem, the fact that your data is reproduced (without a sufficiently detailed explanation that would cause it to be reproduced by any sufficiently capable model) is the problem.
Open AI and friends should be dinged for the fact that they've allowed this to happen, they should pay that liability, and that should encourage them to very strongly police their models and make sure the sort of overfitting is very very rare to non-existent.
But you, nor anyone else, should have what I believe to be the egregious right to say that no one can learn on the data from your work on the internet.
And I understand that's technically not what you're claiming, but when you say that you can deny the right to train models on your data, that's basically what you're saying.
You do have claim to also sue the shit out of GitHub for letting people host your code on their service without your permission, and the guy who originally uploaded it.
So keep in mind here as you respond to me, that I am drawing a line now between properly training a model where it does not overfit, and the example you have where it did produce your copyright material.
> The model is literally trained on my picture; how can it _not_ be a derivative of my work?
In the same way that when a person looks at your code and sees some technique you used and goes oh that's neat I'm going to do that too, that's also not a derivative work.
An AI model does not copy chunks of your code and just sort of haste that code occasionally when it does other things.
An AI model actually learns patterns, builds real working knowledge of connections between things, and uses that knowledge to do new things.
An AI model is going to look at your work and learn things like "oh, I can use an if statement."
You don't have copyright to that. The data contained in your code? The ability to observe facts about how to program? You don't own those. You only own your specific code.
A trained AI model is larger than the sum of its training data. It's not like your code and everyone else's code is sort of bunched into a search engine or something like that, it's actually doing real thinking in there.
And that knowledge? It's just not yours to claim.
> Photoshop analogy is still not really working
It works, again, when you consider what I was talking about above. When the AI model is trained so that it does not overfit.
In the case there is no overfit, the tool doesn't compress your work. It doesn't contain your work. It's just trained on it.
The point of the analogy is to point out that even if you can create something with a tool, you don't own the copyright claim for that tool.
Even with a properly trained AI model, if I were to give it a description of your work, it's probably going to create something that's basically a copyright violation, but that's going to be on the person making that prompt, not the model.
So I think that's where our disparity is. My analogy works because I'm thinking of a model on the average where it doesn't overfit.
You're thinking of the model in the "exceptional" times where it does overfit.
And I do think they are exceptional times. The people making these models really don't want them to overfit so they're using all sorts of techniques to make it happen as little as possible. Clearly not enough, but as a result the average output of these models is actually novel stuff.
So when I see a model I see something more like Photoshop. I see it not as a collection, or a compression of other people's work. Rather, it is something that knows how to write code (or roughly knows the patterns in which words need to be arranged to make code a thing), and there are exceptions within that where you can get it to reproduce what was in its training data.
So when you say this
> Would a very clever compression algorithm suddenly strip from its output the copyright protections of its input?
You're imagining me agreeing to something I don't agree with. I think that when a model reproduces an original work, it's a copyright violation.
But I don't think that abstracts to the whole picture. I do not think that proves training a model on data is compression.
You can deduplicate images that are overfit if you have hundreds of them in your data set, which makes it much less likely overfitting happens.
And the Photoshop analogy _still doesn't work_ even if I were to assume what you suggest, because Photoshop was _not_ built using my artwork. They didn't even look at it while building my artwork. Copilot _is_ built using my copyrighted code. You cannot simply claim that despite being literally built from my code, it is free from my copyright, because that's already assuming the consequent, or what we are discussing here.
Why would their transformation be any different than an advanced compression algorithm? Why would Copilot "know" but another compression algorithm with a 5GB dictionary "not know"? Why should its 5GB dictionary not immediately be a violation since it is built from a lot of copyrighted works? Why would "knowing" magically release from copyright a software built _from_ other copyrighted software? This is already a shitton of assumptions that are not justified in any way, and by driving the conversation with works like "learning", "understand", "know", etc. you are just framing it in a one-sided way.
And the point is that you cannot simply ask those questions about Photoshop itself, because no one's copyrighted artwork was used during its construction. There's no discussion about the merits of Photoshop code "learning" from the artwork because there is no learning whatsoever. To try to jungle both things as the "same" is intentionally misleading.
The entire reason things like "clean room reverse engineering" exist is to allay fears. You don't need clean room to avoid coypright issues, and at the same time it doesn't fully protect you from copyright issues. But if you did use a clean room approach, you at least can demonstrate that you were not looking at the original code when you were implementing your version of it, which is likely to score big points in a trial, _even_ if the code ends up being somewhat similar. Here, Copilot has literally been built using a large corpus of existing copyrighted code, and it stores it with its clearly "super-human photographic memory" (i.e. a huge on-disk database whose size we cannot even estimate). There's simply no way to use a defense like the "clean room" defense. Thus even the _slightest_ similarity to my existing code is extremely likely to be regarded as a copyright violation.
AI code generation systems provide responses (output) to questions or requests by accessing the vast library of open-source code created by developers over decades. However, they do so by allegedly stealing the open-source code stored in virtual libraries, known as repositories.
The authors here are, in turn:
- a lawyer/entrepreneur in copyright-related business (?) and a bunch of random shit like houses made out of mycelium
- the administrator who ran the US Air Force “accelerator” that paid for this research - supposedly an accomplished scientist in many fields but also how likely is it that they significantly contributed?
- an “air force judge advocate” who’s been named to investigate these issues and supposedly writes about technology
- the 2l law student who probably actually wrote the article and did all the work (typical, lol)
All of these people have impressive resumes, but none that make me question my initial assessment that they are going off their gut, rather than looking at what LLMs actually do during training
More importantly, all of this is moot and a waste of resources IMO - the LLMs are already trained well on language, and further improvements will not come from just plugging in a bigger corpus. I also really really don’t think Microsoft will have to crawl GitHub repos to teach GPT6 to program in python 4.0; generative examples, reading the spec, and agential systems make that concern moot.
More gripes because I’m feeling cranky any this paper makes me smile:
The rise of Generative Artificial Intelligence systems (“AI systems”) parallels the Greek myth of Pandora who was overwhelmed with curiosity and opened the Box “[r]eleasing curses upon mankind.” Pandora’s Box is not solely about evil or curses as the artifact-looking Box included Elpis, the personified spirit of Hope, and is a clear reminder that a lot of good can come out of the development of AI systems.
To add some more context to the analogies here, AI systems can be thought of as: “[t]he monster plant Audrey II in Little Shop of Horrors, constantly crying out ‘Feed me!’”Why? Because ChatGPT and other AI systems provide a natural language response (output) to questions or requests by accessing vast libraries of content created over decades.
AI systems trained on code are nothing more than detection calculators or processors that can suggest or simulate statistical patterns, which is certainly not the equivalent of human-like reasoning.
AI systems are certainly here to stay and will replicate as the mythical Lernaean Hydra,Literally the only reason I read this paper was to figure out what a lenaean hydra is, and AFAICT it’s just mentioned this one time in the conclusion. And for all that, the metaphor seems… random? “LLMs are like hydras because we can only kill them through starvation” doesn’t relate very clearly to the actual thesis, which is “the law should mandate that companies pay individuals for the use of their content in AI training sets”
But again can’t really win an argument here either way, so sorry if I offended! Definitely just my anecdotal take
> The terms “person” and “individual” are defined as a natural person, as the term is defined by the United States Patent and Trademark Office (PTO), and/or 35 U.S.C. § 100, as amended. The term “Artificial Intelligence Model” means any non-human generative machine learning system or computer program, algorithm, or functional prediction engine supported by cloudbased/computing platforms. The term “Source Code” means the preferred form of a program for making, creating, and modifying software source code, documentation source, and configuration files.
> No use, modification, combination, study, collection, share, reproduction, distribution, and/or access of Software may be made under this License, by any non-human generative Artificial Intelligence Model without the express written consent of the inventor, which may be withheld or delayed for any reason. Any appropriation, adoption, disclosure, reproduction, use, and/or access of the licensed Software by any non-human Generative Artificial Intelligence Model shall immediately terminate all rights granted to the Licensee. The Licensor shall have the right, at any time, to withdraw consent by written notice, thereby terminating with immediate effect all use of Software made under this License unless otherwise specified. This License is the controlling instrument and supersedes all prior and conflicting Terms of Service, Privacy Statements, and/or Terms for Additional Products and Features of source repositories where this License may be distributed by the owner of the License.
> By accessing and using this data, you acknowledge that you have read, understood, and agree to be bound by these terms and conditions. If you do not agree to these terms and conditions, you may not access or use this data. You may not use this data for the training or inference of Generative artificial intelligence models without the prior permission of the copyright holder. (“Generative artificial intelligence models” are used to create new content or data that is similar to the original data, but not identical. Examples of Generative artificial intelligence models include but are not limited to, text generation models, image and video generation models, and music generation models. The restrictions on Generative artificial intelligence models apply to any use of this data, whether the generative artificial intelligence is trained on this data or uses this data for inference.)
> Any attempt by other artificial intelligence models to access or use this data without such permission shall be deemed a violation of this license and a breach of copyright laws. The copyright holder reserves the right to pursue all legal remedies available, including but not limited to injunctive relief and damages, against any party that violates this license.