> Do anti-LLM types expect to be vindicated in an orgy of copyright lawsuits that resets the industry back to 2022? What do people pushing expect to accomplish?
You weren’t kidding, huh.
You weren’t kidding, huh.
> "The true purpose of AI is to allow wealth to access skill without allowing skill to access wealth."
It COULD be used for good (that's why its proponents use tone-deaf analogies comparing AI to seats on a rocket)... but we know that --- for the most part --- it WON'T be used for good... (It's already being used to spread more disinformation and to fan the flames of fascism).
Technically, you could argue that shell corporations could protect journalists, but you don't see journalists destabilizing democracies by fueling dark money to alt-right groups here and there.
Works great for them because whenever something goes wrong they can just blame "outdated data" and move on.
For 99.999% of people, it is literally kthxbye on all code they execute on all their devices.
I'm not "blindly trusting" code on my computing devices. I'm trusting the vendors / maintainers to do their job.
Until very recently the norm has been that the vast majority of code had human eyes and hands on it.
Edit:
The owners of those human eyes and hands had some type of accountability (either reputationally, in the case of free/open-source software, or occupationally, in the case of proprietary software).
The LLM has no accountability as to the output it generates.
The companies who make the LLMs also seem to have very little accountability, too. We've assumed a "blame the victim" stance when people use LLM-generated output in some inappropriate ways (legal briefs with "hallucinated" citations, articles "written" by LLMs). Whether that's the right location for accountability to be placed isn't for me to say, but that seems to be how it is.
I'm not sure that we're applying accountability to LLM-generated code in the same way we are for, say, the LLM-generated legal brief.
LLMs, at least as they're currently constructed, aren't deterministic (the whole "temperature" thing). I don't see how to build a mechanistic test for something that has non-deterministic output. It feels a little bit like solving the halting problem.
I have no doubt we'll move away from human code review. The idea of large amounts of software edifice being built upon foundations that no human has reviewed or, perhaps even understands, is horrifying to me, though.
Also this sort of 'technological whataboutism' really isn't helpful, compilers are entirely different from LLMs. I agree that it doesn't make much sense to read or review LLM output in detail, but I also don't plan to use LLM output for anything important or mission critical. That would be irresponsible.
No such limits for LLMs where losing all your files is about par for the course for everyone who uses them regularly.
The actual machine code depends on several parameters, and it is very hard to replicate them, hence why many devs get benchmarks with JITs wrong.
Additionally, compiler optimisation passes with machine learning is starting to be a thing, yet another way how the machine code differs for the same input across compiler executions.
What quality does it have that humans had eyes on code?
Elite teams likely still produce code of better quality with a higher qa bar than agentic code. But that category is dimishing everyday.
The core point is that so much trust is reduced to "Joe in cubicle". Also a lot more than what he can carry.
Most of the code that is being executed on your behalf is far from written by elite teams.
It's too early to say whether LLM generated projects will ever reach that sort of maturity, most examples I've seen so far are basically "fire and forget". But lets talk again in one or two decades, maybe there will be counterexamples of successful open source projects which will be just as well llm-maintained as human-mainained.
But I suspect that to reach that sort of maturity, the resulting human effort will be mostly the same (e.g. not much of a productity win - except maybe on the 'edges', e.g. maintaining the test suite, documentation, helping to analyze bugs..., e.g. these are examples where LLMs are genuinely useful and where plagiarism hardly matters).
There are some areas that are critical and where software developers carefully will detail stear the work.
But the vast amount of software written, react components and rest endpoints, are very ripe to be entirely written by agents.
In that I agree, nobody should be forced to write React code manually, that's almost a human rights violation ;)
REST endpoints (and the code talking to those endpoints) should be code generated anyway though, no need for LLMs, and instead of human language prompting, a precise IDL should be the spec and basis for a mechnical code generation process. That problem was solved decades ago with much more pedestrian technology.
E.g. it basically comes down to "it's fine to use LLMs for software that shouldn't have been written in the first place", and funny enough that's where LLMs are really good at: creating software that has been written a million times before with only minor variations, and doing this type of work manually (cranking out one cookie cutter React webpage or REST API after another) is essentially what's called 'bullshit jobs' (which bring food on the table though, but that's another topic).
PS:
> and where software developers carefully will detail stear the work.
...I think the further a project evolves, the less this "detailed stearing" will be any more productive than doing the same without LLMs. The older a project, the more the work shifts from implementation to decision making, and in most cases the result of that decision is just a very tiny code change. I already see cases in my daily work when I use 'agentic workflows' where a tiny change takes longer and involves more 'collatoral updates' then just fixing that one frigging line of code by hand like in the olden days, and for LLM-generated code bases I really do prefer to not mix LLM and manual work, I think that's the worst of all options.
The human who had their eyes and hands on the code has accountability.
I think how accountability is going to work, in the case of LLM-generated code, is still an open question.
(I dropped-on an edit to the parent comment to this effect, too.)
They are not.
You can not go go back to a laid off person / one who quit and keep them accountable.
So this is absolutely not the case.
Devs are not accountable for their code.
The company that employed the developer ultimately holds the accountability in the marketplace. The employed developer maintains (or loses) their job because of their accountability to their code (or, at least, they should). There's an economic incentive for all parties involved.
In the free/open-source world the incentives aren't economic, but they're still there.
> or, at least, they should
Seems like you are the obtuse one here, and you appear to know it.
I give up. I feel like you're a robot designed to waste my time.
You are talking from an ideal point of view. You want the companies to hold employees responsible.
But that is is not how it work. Why you appear obtuse.
Eg. Look at th3 therac25 case. No developers was held accountable.
We have spend more than a decade remove accountability from indiviauls. Using limited liability, insurance, and workers protection.
Accountability is the last reason why we need humans over agents.
At the start of this you said: "For 99.999% of people, it is literally kthxbye on all code they execute on all their devices."
I think that's inaccurate. The vast majority of code running on "all their devices" is code made by employees of companies being held accountable through traditional industry methods, or free.open source projects where reputational integrity was at stake. Those developers have been held accountable, for some value of accountable.
Maybe there's less value in human accountability than I think there is. Only time will tell. That's a different conversation.
The code running "for 99.999% of people" is not "literally kthxbye" LLM-generated code without someone behind it holding accountability. Maybe it will be in the future, but it's not now.
Off the cuff, I would be surprised if the GNU project embraced AI, so I'm confused that people think so strongly otherwise.
I am not saying this is not what will happen — the actual law seems to be still up in the air. But if it does happen it will be an existential threat to the GNU and the whole free software ecosystem.
1. There is no indication that is at all likely except for purely vibe-coded projects. It seems highly unlikely and in some countries (e.g. the UK) the law clearly says otherwise.
2. There have been quite a few rulings in countries where it is unclear, and they all set some level of human input that will make AI generated code covered by copyright. Look at the cases that have been in HN stories about cases in the US, Germany and Japan, for example.
2. It would have to be all AI generated, and you would need to replace all the human written parts. Not a practical problem for a large, old project.
If this is their real reasoning they are jumping at shadows. However, this might be like where, the copyright (which is the explanation given in the ToS) is not the real reason (which was explained in the subsequent blog post).
It is interesting that proprietary software businesses, who have an even stronger interest in ensuring their software is covered by copyright in all countries seem to be quite happy to use LLM generated code. Microsoft and many others boast about how much of their code is now LLM generated.
I don’t think taking copyright off the table harms the practice of sharing code. They will still try to use trade secrets to restrict code sharing and contracts, but using GNU software won’t be stopped. It will reduce the ability to sue people not sharing their modifications but that was always outside the mainstream, and places like AWS, Apple, and Google find ways around it anyways since it doesn’t cover hosted services or non-linked code.
The core stream of openly developed and exponentially improving software does not need copyright to win if it cannot be sued for copyright violation.
Now I suppose some OpenAI lawyer is trying to find a way to sue humans for copyright infringement while keeping them safe from lawsuits, so we can worry about that attack.
In images it's much more _obvious_, but I think code is very likely to have similar problems. Like, websites that an LLM spits out are often very very similar. It wouldn't be shocking to me if some of the code in the training set was trained off GPL code, and there are small GPL violations all over the place.
By the way, do you have a source on Midjourney spitting out copyrighted stuff all the time? Does it happen at random or when users intentionally steer the prompt in that direction? I suspect it's the latter but I admit I'm not really familiar with this tool.
Its much the same as someone creating a fork of GPL code in which they make additions that they put in the public domain. All the original code and the fork as a whole would remain GPL.
its not a small risk, its a negligible risk.
As you pointed out yourself, there's always the option to create a fork that does allow AI contributions, which may eventually force a re-assessment of the policy if the gap in utility grows too large.
That would take a very long time if contributions are reviewed etc. By then any legal ambiguities would be clear.
> a public domain codebase with some GPL code
which would still be a GPL codebase
> One way to do so is to outright ban contributions leveraging tools that are able to generate public domain code at superhuman speeds.
Can they generate code that would pass the quality standards, and pass the processes, of a project like this at superhuman speed? There is a separate requirement that contributors must be able to understand code and answer questions about it so a human would have to review code before even trying to contribute it.
> As you pointed out yourself, there's always the option to create a fork that does allow AI contributions, which may eventually force a re-assessment of the policy if the gap in utility grows too large.
1. if you are right that LLMs will do well enough to create a huge gap, then that is inevitable. 2. if you are wrong about that then it is unnecessary to try to stop it.
> which would still be a GPL codebase
If the GPL-licensed parts have become so insignificant that they can easily be replaced, it effectively no longer would be.
LLMs fundamentally hinder all three. I'm not sure you even need to look much further than that.
> In the kernel community we do open source because it results in better technology, not because of religious reasons.
> And so we make decisions primarily based on technical merit. Not fear of new tools.
You are not making an argument based on technical merit here.
1) Courts reverse their previous decisions and declare LLM generated code as belonging to LLM labs.
2) LLM labs decide to assert their copyright and sue open source projects.
3) They are able to prove that the code was generated by an LLM and not just any LLM but their LLM.
The concern instead is that LLMs and all of their outputs may be found to be derivative works of their entire training set, and thus rendered unusable (as the training set is not distirbutable under any license).
I think this ship has long sailed and no court is going to dare give such a decision given the money involved, for better or for worse. But it's a much more realistic scenario, in principle, than LLM labs going mad and attacking their own customers.
Edit to add: there is another, completely different, copyright risk associated with LLMs - and one that is much more realistic. It is the fact that code generated by LLMs may not, in fact, be copyrightable at all. Which would mean that it can't be subject to the GPL. As long as it remains a minority of GCC code, this wouldn't matter much, but it could in time lead to significant portions of GCC becoming public domain, and thus cooyable, modifiable, and redistrubutable without providing the four freedoms.
What is likely to get more muddy over time is the accuracy of any copyright registration, and the enforcement of copyright infringements on portions of the whole. These are already complicated cases and definitely so for compilers with so much "scènes à faire".
It's not clear how much this has a negative impact on cases around the whole, which tend to be the more important cases for the four freedoms that, while they have other intentions, have a primary intention of ensuring that the whole continues to be available for redistribution and extension in perpetuity.
I do not think that there is a clear link between these two areas at all, and the GPL's most important intents may be far safer long term than concerns of dilution suggest.
I do believe though that, if the LLMs were found to be derivative works of their training set, it would follow almost directly that their output is also a derivative work of that same training set - given how these LLMs operate. And even if the liability fell with the LLM providers (which may not be so clear cut for, say, local models, fine tuning, etc), that would still mean everyone would have to excise any LLM generated content they are distributing.
I doubt so. Let's say Harry Potter is in the training set and you ask the LLM to generate a quick sort function in C, is that quick sort function a derivative of Harry Potter? What if you ask the LLM to output some known public domain work? That leads to a contradiction where according to one definition, the work is public domain and according to the other, it is a derivative of Harry Potter. It seems to me that there's no other option but to consider each output on its own merit.
Furthermore, I don’t think you can really assume that the courts will rule a certain way on this just because of the money involved; there’s a lot of money involved when it comes to the copyright holders too, and they’ve long enjoyed a rather favorable status with the courts and legislators. (For example, in the days of P2P file sharing lawsuits and attempts to legislate P2P file sharing, the software industry was already many times the size of the media industry, but the media industry consistently won.)
I don't think this is all that plausible, even though I agree with you that it's not settled law. The size of the AI industry is gigantic, and a ruling that they are infringing the copyright of every piece of content in their training set would essentially shut them down entirely. Such a decision, if final, would probably easily wipe out a few hundred billion dollars on the stock market. Even if any court was willing to go that far, almost certainly lawmakers would step in and modify copyright law to prevent this from happening - both in the USA and the EU.
I don't think there is any comparison to make with the file sharing battle. That was a much, much smaller industry, it was not a significant chunk of the total hardware and software industries. Plus, the software titans were not nearly as well connected politically as they are today.
The second thing is that I’m not necessarily talking about whether _a specific LLM itself_ infringes copyright, but whether _its output_ is covered by the copyright of _its training material_. Whether training an LLM is an activity that infringes copyright is not well-settled in any precedential way, whether the trained LLM as an artifact infringes copyright is even less settled, and whether the output of that LLM is either infringing or covered by copyright is also not settled. These are all still extremely open questions.
That means anyone doing reasonable risk management should not just blithely race ahead and assume that there’s no infringement, which appears to be the approach the GCC project is taking explicitly and which also appears to be the approach projects like Linux and LLVM are taking implicitly (mostly through weasel-language like accepting responsibility for code you’re submitting).
"To summarize the analysis that now follows, the use of the books at issue to train Claude and its precursors was exceedingly transformative and was a fair use under Section 107 of the Copyright Act. And, the digitization of the books purchased in print form by Anthropic was also a fair use but not for the same reason as applies to the training copies. Instead, it was a fair use because all Anthropic did was replace the print copies it had purchased for its central library with more convenient space-saving and searchable digital copies for its central library — without adding new copies, creating new works, or redistributing existing copies."
https://fingfx.thomsonreuters.com/gfx/legaldocs/jnvwbgqlzpw/...
IANAL and don't know how significant this decision is, but it is, at the very least, how one judge views it.
Personally, I don't think judges will rule a certain way because of the money involved but because it seems clear that training a ML model is highly transformative.
Anthropic is trying to settle the case with most plaintiffs with respect to obtaining their works in an infringing way, but there are still plaintiffs pursuing the case on both the grounds that the remedy is insufficient (being only about $3000/work, when it has been as high as $250K/work in other copyright infringement cases and via statutory damages) and also on the grounds that the ruling that training is fair use was an error of law on the district court judge’s part.
Notably it doesn’t cover whether the output of the trained LLM continues to attach the training set’s copyright, which is independent from whether the training itself was an infringing activity. And there’s a substantial argument that the judge erred, if it can be shown that the training works are stored in a recoverable manner (even with some loss/defredation) rather than more extensively transformed.
1) Someone re-licenses GCC under a non-GPL license.
2) EFF sues them, to stop the behaviour
3) Court tells EFF that they have no standing to sue because LLM generated content has no copyright
Obviously this happening would be in the future after someone translated GCC to Rust with LLMs or something.
1) All of GCC would have to be LLM generated. If some parts are not, it's sufficient to prevent the re-licensing.
2) That someone would have to prove that all the GCC code was in fact LLM generated. Good luck doing that.
3) A court would have to decide that all the LLM-generated code does in fact fall under public domain, because it involved insufficient human input.
A pointless act since code is now free. The GPL exists to ensure code freedom in an era when code was expensive. Yes I'm aware that the meaning of "free" is a bit conflated here, but the point stands.
IP law (like a lot of other things) has been skewed toward the interests of business, even when that conflicts with fairness or societal good. For all its flaws (IMO), the free software movement tends to be principled. Just because something is legal doesn’t mean it’s right.
News back then were about intentionally prompting to output known copyrighted material.
The parent comment still stands in my opinion:
When, despite millions of developers using agentic AI already, are these lawsuits supposed to manifest?
> News back then were about intentionally prompting to output known copyrighted material.
First, there are other cases if you take the time to dig. This is quite an old example (GPT-2) as i haven't kept up to date on this field recently, but it does show that this problem has been known about since before these systems were widely adopted: https://arxiv.org/abs/2012.07805 [0]
Second, GP said nothing about the type of effort required to make it happen, just that it can be done and that the copyright owner could come along and cause legal problems later. It's absolutely possible to have a fly-by contributor who purposefully asks for code that reproduces X/Y/Z without a maintainer knowing about it.
But then the maintainer is the one in legal trouble.
> When, despite millions of developers using agentic AI already, are these lawsuits supposed to manifest?
Legal / copyright / etc. cases often take a lot longer than a couple of years to come to fruition.
---
[0]: edit -- to clarify this is an example of the reproduction problem, not an example copyright infringement case.
The concern discussed here is copyrighted material being generated unintentionally and the original author asserting their rights.
This has, to my knowledge, not happened once.
If we are not talking about unintentional violations, I don't understand the point of the discussion.
I can also intentionally copy paste the copyrighted material into my merge request without the use of AI in an attempt to get the maintainer into trouble.
both intentional (malicious contributor) or unintentional (Large-Laundering-Model) are copyright issues -- which is the point of GCC's policy.
> I can also intentionally copy paste the copyrighted material into my merge request without the use of AI in an attempt to get the maintainer into trouble.
You can. You can also do it significantly faster with significantly less effort while being harder to detect using agents etc.
That is obviously not what anyone was referring to, nor does it make sense, when there is a much more reasonable basis to prohibit the same contribution.
Namely inserting vulnerabilities. This one actually happened before afaik, and provides a clear benefit to the attacker.
If a contributor doesn't care about submitting copyrighted code, they can do it without an LLM as well.
Plenty of github accounts now are agent-driven monstrosities just trying to inflate someone's contribution stats etc.
Someone tried to contribute “vibe-coded” device support to a project I’m involved with, they said they did it all based on the device documentation, the code their agents spit out was copied verbatim out of a (GPL’d) project with which I’m familiar which supports that device.
LLMs are not learning things and then using that learning to construct new things. They are essentially a form of lossy compression of their training set. And you don’t need to be explicit about trying to reproduce a portion of that training set for an LLM to output one.
I am not aware of any study attempting to measure unintentional reproduction.
With your example, I question whether you have seen this happen first hand. For all I know, the contributor could have explicitly prompted the model to reference the GPL project and had the agent clone the code from the web.
I would be surprised if a frontier model generated unexpected copyright headers during typical usage.
the "entire history of" is circa 3-4 years, which is very much a tiny period of time compared to normal legal system / copyright law stuff (IANAL).
alternative perspective: it's just taking time for the lawyers to figure out what they can sue them for.
It’s a hard balancing act to do. Give in too much randomness and you get non-sensical outputs that are difficult to align. Fit too closely to the training data and the model regurgitates the training data.
And oh, what’s that copyrighted material we never made any agreement to use doing in there?
> The complaint argued that "the basis of the Gaye defendants' claims is that "Blurred Lines" and "Got To Give It Up" "feel" or "sound" the same. Being reminiscent of a "sound" is not copyright infringement. The intent in producing "Blurred Lines" was to evoke an era. In reality, the Gaye defendants are claiming ownership of an entire genre, as opposed to a specific work"
they lost (eventually) https://en.wikipedia.org/wiki/Pharrell_Williams_v._Bridgepor...
wider point -- whether or not a copy is a copy and whether it is is infringing on copyright or not ultimately has to be decided by a court case when it's not an obvious and clear cut violation. especially in the USA with the utterly mental fair use law.
He's correct but it's an irrelevant argument, he's simply making an emotional, and very childish, attack on someone because they don't like the tech he likes. Banning LLMs in your project, regardless of the reason, does not mean you think they are ever going to go away, or even that you want them to.
How do you build a legally-sound product using an LLM that has been successfully sued for violating copyright in X countries around the world?