GitHub Copilot as open source code laundering?
twitter.com
twitter.com
As GitHub is a Microsoft company and OpenAI although a non-profit just got a massive one billion investment from Microsoft (presumably not for free), will it start spitting out once in a while Windows kernel code ? :-)
And if it was NOT trained on Microsoft source code, because it could starting suggesting some of it...Is that not a validation that the results it produces are a derivative work based on the work of the open source code corpus it was trained on ? IANAL...
There is plenty of leaked Windows source code on Github, so chances are that co-pilot would give quite good suggestions for implementing a Win32-compatible kernel. Then watch and see if Microsoft will try to argue that you are violating their copyright using code generated by their AI.
“GitHub included Microsoft proprietary code in the training set because they view the results as non-derivative” and “GitHub didn’t include Microsoft proprietary code because they view the results as derivative” are clearly not the only options. They could have not included Microsoft internal code because it was way easier to just use the entire open source corpus, for example.
They don't claim they used an “open source corpus” but “public code” because such use is “fair use” not subject to the exclusive rights under copyright.
They don't claim it wouldn't be a license violation, they claim licensing is irrelevant because copyright protection doesn't apply.
> And if it was NOT trained on Microsoft source code, because it could starting suggesting some of it...Is that not a validation that the results it produces are a derivative work based on the work of the open source code corpus it was trained on ?
No, that would just show them to not want to expose their proprietary code. It doesn't prove anything about derivative works.
Also, their own claim is not that the results aren't a derivative work but that training an AI is fair use, which is an exception to the exclusive rights under copyright, including the exclusive right to create derivative works.
Literally people need to quit Microsoft and join Github to take a role at Github.
They explicitly state "public" code so the answer is most certainly "no".
The underlying question is whether the output is a derivative work of the training set? Sidestepping similar issues is why GCC and LLVM have compiler exemptions in their respective licenses.
It wasn't trained on internal Microsoft code because the training set is publicly available code. It has nothing to do with whether or not it suggests exactly identical, functionally identical, or similar code. MS internal isn't publicly available. Copilot is trained on publicly available code.
The question (and implication) is: why not train it on MS internal code, if the claim that the output isn't license-incompatible is true.
If the output doesn't conflict with any open-source license (ie. it springs into existence from general principles, not from "copying" licensed code -- then MS-internal (in fact, any closed-source code) should be open-season.
I can imagine a few of the non-obvious segments of code I've written being "recognizable" methods to solve certain problems. And, they are certainly licensed (GPL + Commercial, in my case).
I think, at the very least, that a set of AIs should be trained on different compatible sets of code, eg. GPL, AGPL, BSD, etc. Then, you could select what amount of license-overlap is compatible with your project.
As an example, my grandfather (an old school EE who got his start on radar systems in the 50s, who then got his radiology MD when my jewish grandmother berated him enough with "engineer's not doctor though...") has some really cool patents around highlighting interesting parts of the frequency domain in MRIs that should make detection of cancer a whole lot easier. As an implementation he did a bunch of tensor calculus by hand to extract and highlight those features because he's an incredibly smart old school EE with 70 years experience cranking that kind of thing out with only his trusty slide rule. He hasn't gotten any uptake from MRI manufacturers, but they're all suddenly really into recurrent machine learning models to highlight the same sorts of stuff. Part of me wants to tell him to try selling it as a machine learning model and just obfuscate the fact that the model was carefully hand written rather than back propagated.
I'm personally pretty anti intellectual property (at least how it's implemented in the states), but a system where large entities that have the capital investment to compute the large ML models can launder IP violations, but little guys get stuck to the letter of the law certainly seems like the worst of both worlds to me.
How many models are back-propagated first and then hand-tuned?
Any chance some fantastic HNer could chime in there?
Two examples I can think of are doing linear regression on the square of your input. For deep learning, people have improved visual representation by taking samples of the colors at various frequencies. [1]
being able to produce valid code is not the bottleneck of any developer effort. no projects fail because code can't be typed quickly enough.
the bottleneck is understanding how the code works, how to design things correctly, how to make changes in accordance with the existing design, how to troubleshoot existing code, etc.
this tool doesn't make anything any easier! it makes things harder, because now you have running software that was written by no one and is understood by no one.
That doesn't help anyone!!
I am usually pretty pro-Microsoft, but this tool is a security nightmare and a bad idea all around. It will cause many (most? all?) who use it far more work than it saves them, long-term.
I use the React plugin for Webstorm to avoid having to write the boilerplate for FCs. Maybe in the future Copilot will replace that usage.
I think we should strive to improve our programming languages to make less of this boilerplate necessary, not to make generating boiler plate easier. The latter is just going to make software less and less wieldy. Imagine the horror if instead of (relatively) higher level programming languages like C we were all just using assembly with code generation.
I really like your point on symptoms of insufficient abstraction. I do worry that we always see abstraction as belonging in language. Which in turn we treat as a precious singleton, and fight about.
At least in my own hacking, I'm surprised how infrequently I see programmers write programs that write programs. I'm surprised how infrequently I see programmers programming their shell, editor, or IDE.
What we really need aren't tools that help us write code faster, but tools that help us understand the design of our systems and the interaction complexity of that design.
- Is GitHub actually going to pay any attention to that, or are they just going to ingest the code and thus violate its license anyway?
- If they go ahead and violate the code's license, what are the legal repercussions for the resulting model? Can a model be "un-trained" from a particular piece of code, or would the whole thing need to be thrown out?
That effectively “overrides” any license or term that you’ve specified for your repository, since you’ve already licensed the content to GitHub under different terms. Of course, people who are not GitHub are beholden to the terms you specify.
[1] https://docs.github.com/en/github/site-policy/github-terms-o...
> We need the legal right to do things like host Your Content, publish it, and share it. You grant us and our legal successors the right to store, archive, parse, and display Your Content, and make incidental copies, as necessary to provide the Service, including improving the Service over time. This license includes the right to do things like copy it to our database and make backups; show it to you and other users; parse it into a search index or otherwise analyze it on our servers; share it with other users; and perform it, in case Your Content is something like music or video.
But, it goes on to say:
> This license does not grant GitHub the right to sell Your Content. It also does not grant GitHub the right to otherwise distribute or use Your Content outside of our provision of the Service, except that as part of the right to archive Your Content, GitHub may permit our partners to store and archive Your Content in public repositories in connection with the GitHub Arctic Code Vault and GitHub Archive Program.
I'm not a lawyer, but it seems ambiguous to me if this ToS is sufficient to cover CoPilot's butt in corner cases; I bet at least one lawyer is going to make some money trying to answer the question.
On the extreme end, "analysis" is so broad that it could arguably cover breaking down a file of code into its constituent methods and just saving the ASTs of those methods verbatim for Copilot to regurgitate. That's obviously not an acceptable outcome of these terms per se, but arguably isn't any different in principle from what they're already doing.
Ultimately, as I understand, courts tend to prefer a common sense outcome based on a reasonable human understanding of the law, rather than an outcome that may be defensible through some arcane technical logic but is absurd on its face and counter to the intent of the law. If a party were harmed by an instance of Copilot-generated copyright infringement, I don't see a court siding with this tenuous interpretation of the ToS over the explicit terms of the source code license. On the other hand, it would probably also be impossible to prove damages without something like a case of verbatim reproduction, similarly to how having a developer move from working on proprietary code for one company to another isn't automatically copyright infringement.
I doubt that GitHub is doing anything as blatantly malicious as copying snippets of (GPL or proprietary) code to explicitly reuse verbatim, but if they're learning from license-restricted code at all then I don't see how they wouldn't be subjecting themselves and/or consumers of Copilot to the same risk.
I do not upload my code to github, or give them any special permissions, and I am confident my code was included in the model's corpus.
From the definitions section in the same doc:
> "Your Content" is Content that you create or own.
That will definitely exclude any mirrored open-source projects, any open-source project that has ever migrated to Github from another platform, and also many forked projects.
So if someone uploads a Hollywood movie to Youtube, Youtube doesn't get the rights to play that movie from them because they didn't have the rights in the first place. Of course, if the actual copyright owner uploads it, it's now permissible for Youtube to play it, even if it's the copy that someone else provided. [This has torpedoed a few filesharing lawsuits.]
That's nonsense because they could claim that for almost any reason.
E.g. assume Google put the source code of Google search in Github. Then Github copies that code and uses it in their own search, since that "improves the service". Would that be legal?
It's like selling a pen and claiming the rights to anything written with it.
The person uploading files to github is also not necessarily doing so with permission from the rights holder, which might be a violation of the terms of service, but would mean there's no agreement in place.
Then, let’s say the AI generates some new code for someone, and it is nearly identical to some bit of code that you wrote in your project.
If they didn’t use your code in the model, then the generated code is clearly not a copyright violation, since it was effectively a “clean room” recreation.
If your code was included in the model, is it therefore a violation?
But then again, it comes down to how can someone prove their code was included or not?
What if the creators don’t even know? If you wrote your model to say, randomly grab 50% of all public repos to use in the model, then no one would know if a specific repo was used in the training.
I suppose that for most open source licences this at the very least involves attribution for all the people that produced the code that the program was trained on ?
"All rights reserved" makes sense on final items, like books or physical records, that require no copy or change after owner-approved manufacturing has taken place. It doesn't really make sense on digital artefacts.
Also, in your example, the copyright for the book or dvd is for the content, not the physical item. You can do anything you want with that item but not the content. My code is similar, I'm licensing my provider to serve you a visual representation of the files so you can experience the content, not giving you a license to run that code or use it otherwise.
I do wonder, though, if GPL owners worried about their code being shanghaied for this purpose could file arbitration claims and exploit some particularly consumer-friendly laws in California which force companies to pay fees like when free speech dissidents filed arbitrations against Patreon.[0] Patreon is being forced to arbitrate 72 claims individually (per its own terms) and pay all fees per JAMS rules. IANAL, so I don't know the exact contours of these rules, or if copyright claims could be raised in this way, or even if GitHub's agreements are vulnerable to this loophole, but it'd be interesting.
[0]https://www.dailydot.com/debug/patreon-suing-owen-benjamin-f... (see second update from July 31).
Under the right circumstances, Copilot will recite a GPL copyright header. It isn't a huge step from that to some other commonly repeated hunk of GPLed code -- I'd be particularly curious whether some protected portion of automake/autoconf code shows up often enough that it'd repeat that too.
Somehow p-hacking springs to mind
The issue is with the users of Copilot potentially violating copyright and licences (non-attribution for instance) and with Microsoft facilitating it. (See also : A&M Records, Inc. v. Napster, Inc.)
By the way, code generated by Github co-pilot is likely incompatible with Microsoft's Contribution License Agreement [1]: "You represent that each of Your Submission is entirely Your original work".
This means that, for most open-source projects, code generated by Github co-pilot is, right now, NOT acceptable in the project.
[1] https://opensource.microsoft.com/pdf/microsoft-contribution-...
For this scenario, how is using Co-Pilot generated code different from using code based on sample code, Stack Overflow answers, etc.?
My point is: when I'm copying code from a source with an explicit license, I know whether I'm allowed to copy it. If I pick code from co-pilot, I have no idea (until tested by law in my jurisdiction) whether said code is public domain, AGPL, proprietary, infringing on some company's copyright.
[1] https://stackoverflow.com/legal/terms-of-service#licensing
I have recommended as such to the CTO and other senior engineers at the startup I work at, pending some clear legal guidance about the specific licensing.
My casual read of Copilot suggests that certain outputs would be clear and visible derivatives of GPL code, which would be _very bad_ in court- probably? Some other company can have fun in court and make case law. We have stuff to build.
If it's spitting out verbatim code 0.1% of the time, surely it's spitting out copied code where only trivial things are different at a much higher rate.
Trivial things meaning swapped order where order isn't important, variable/function names, equivalent ops like +=1 vs ++, etc.
Surely it's laundering some GPL code, for example, and effectively removing the license in a way that sounds fishy.
Whether or not the result is a license violation is tricky legal question. As always, IANAL.
It seems to me an important question is, "is this like a human who learns from examples, or is this really a derivative work in the copyright sense?".I'm not sure how to answer that. I'm not a lawyer. I don't know if many lawyers can answer that question either!
I don't have photographic memory, so I largely don't memorize code. I learn general techniques, and memorize simple facts such as APIs. I can memorize some short snippets of code, but these probably aren't enough to be copyrightable anyway.
> The type of model they use isn't retrieving
How do we know? It think it's very likely that it is largely just retrieving code that it memoized, and doing minor adjustment to make the retrieved pieces fit the context. That wouldn't differ much from finding code that matches the problem (whether on SO or Github), copy pasting the interesting bits, and fixing it until it satisfies the constraints of the surrounding code. It's impressive that AI can do that, but it doesn't sound like it's producing code.
I think the alternative to retrieving would actually require a higher level understanding of the world, and the ability to reason from first principles; that would be much closer to AGI.
For example, if I want to implement a linked list, I'm not going to retrieve an implementation from memory (although given that linked lists are so simple, I probably could). I know what a linked list is and how it works, and therefore I can produce working code from scratch.. for any programming language, even ones for which no prior implementations exist. I doubt co-pilot has anything remotely as advanced as this ability. No, it fully reliant on just retrieving and reshaping a pieces of memoized code; it needs a large corpus of code to memoize before it can do anything at all.
I don't need a large corpus of examples to copy, because I use my ability to reason in conjunction with some memoized general techniques and common APIs in order to produce original code.
wanna see the source code of my AI model? oh, it's closed source
it's just coincidence that nearly 100% of my future linux-like kernel code looks the same as linux the kernel, bear in mind that my closed-source AI model takes inspiration from GitHub Copilot, there is no way that it will copy any source code
Well, they were ignored and this is the result. A for profit company built a proprietary system using every code hosted in its platform without respecting the code license.
There will be a lot of people saying this is not a license violation but it is, and more, it is an exploitation of other people work.
Right now I'm asking myself when people will stop supporting these kind of company that exploit people's work without giving anything in return to people and society while making a huge amount of profit.
If we read a book and use its instructions to build a bicycle, is it an exploitation of people's work?
No, no it's not.
When you read a book and copy this book partially or entirely to create a new book or create a derivative work using this book without citation it's called plagiarism and copyright infringement. It is not only exploitation, it is against the law.
If you feed an entire library to an AI to generate new books without source citation and copyright agreements it is not only exploitation, it is against the law. We can call this automated plagiarism and copyright infringement, and automated or not, it is against the law. Except if you use public domain books. It wouldn't be illegal but highly unethical considering there are powerful companies with big pockets bending public domain's laws to avoid their assets to be public available (I'm looking at you Disney), but that is another story.
Of course we generate the world around us and its rules but I get angry every time we compare people to machines and say that it's the same thing. No it's not. We are constrained by time and space. I can't add more brain or more eyes to my body so I read more books can I? Microsoft can have a small city of servers somewhere and that could replace lots of people's jobs.
Thinking that this would conveniently bypass the fact that your goal was to copy the code seems to be the most common legal fallacy amongst software developers. The law will see straight through you, and you will be found to have infringed copyright. The reason is well explained in "What Colour are your bits?" (https://ansuz.sooke.bc.ca/entry/23).
EDIT: to I can write my worry, semi-jokingly, as a conspiracy theory: Microsoft is using thousands of unsuspecting (and unwilling) developers to turn a huge copylefted corpus of algorithms into non-copylefted implementations. Even assuming that developers that use the co-pilot use non-copyleft licenses only 50% of the time, there's still a constant trickling of un-copyleftization.
You'd just have to wrap it in a nice complex model representation so it's a black box you fed example OS's with some meta-data into and it happens to output this very useful data.
After all, once you use something as input to a machine learning model apparently the license disappears. Sweet.
* Someone leaks Windows 10/11 source code
* Copilot picks it up in its training data
* Someone uses copilot to generate a Windows clone and starts selling it
I wonder how Microsoft would react to that. I wonder if they've manually blacklisted leaked source code from Windows (or other Microsoft products) so that it doesn't show up in Copilot's training data. If they have, that means Microsoft recognizes the IP risks of having your code in that data set, which would make this Copilot thing not just the result of poor planning/maybe a little incompetence, but something much more devious and malicious.
If Microsoft is going to defend this project, they should introduce all of their own source code into the training data.
why do you think it has to be source code? it could be the compiled code after all.
If what we're talking / fantasizing about here works in the way of `let x = 42` it should equally well work with `loda 42` &cpp, so source code be damned. It was ever only to be an intermediate step, inserted between the idea and the working bits, to enable humans to helpfully interfere. Dispensable.
You could test this with one of Microsoft's products that is already on GitHub - like VSCode. I doubt you would get anywhere with just copilot.
It does add some degree of plausible deniability (accidental violation, instead of intentional), but I don't think it would matter much.
// The following code implements the functionality of <popular GPL'd library>
3) Delete top comment if necessary :P
(It's pretty unlikely that this will actually work but the approach could well do.)
Why not just copy it and then edit it? If a snippet is changed both logically and syntactically to not resemble the original, then it’s no longer the original and you aren’t in any licensing trouble. There is no meaningful difference between that manual washing and a clean room implementation. All the ML changes here is the accidental vs deliberate. But it will be a worse wash than your manual one.
If you tell it to generate "a function for calculating the barycentric coordinates of a ray-triangle intersection", you might get a working implementation of a popular algorithm, adapted to your language and existing class/function/variable names.
But if you tell it to generate "a smartphone operating system", it probably won't work...and if it does, it would most likely use giant chunks of Android's codebase.
And if that's true, it means that copilot isn't really generating anything. It's just a (high-tech) search engine that knows how to adapt the code it finds to fit your codebase. That's still a really cool technology and worth exploring, but it doesn't do enough to justify ignoring software licenses.
But since now APIs are unprotected you could feed it all of the class structure and method signatures to have it fill in the blanks. I don't know if that gets you a working operating system but it seems like it will get you quite a long way
> copyright does not only cover copying and pasting; it covers derivative works. github copilot was trained on open source code and the sum total of everything it knows was drawn from that code. there is no possible interpretation of "derivative" that does not include this
Copyright law is very complicated (remember Google vs Oracle?) and involves a lot of balancing different factors [0]. Simply saying that something is a "derivative work" doesn't establish that it's copyright infringement. An important defense against infringement claims is arguing that the work is "transformative." Obviously "transformative" is a subjective term, but one example is the Supreme Court determining that Google copying Java's API's to a different platform is transformative [1]. There are a lot of other really interesting examples out there [2] involving things like if parodies are fair use (yes) or if satires are fair use (not necessarily). But one way or another, it's hard for me to believe that taking static code and using it to build a code-generating AI wouldn't meet that standard.
As I said, though, copyright law is really complicated, and I'm certainly not a lawyer. I'm sure someone out there could make an argument that Copilot is copyright infringement, but this thread isn't that argument.
[0] https://www.nolo.com/legal-encyclopedia/fair-use-the-four-fa...
[1] https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_...
[2] https://www.nolo.com/legal-encyclopedia/fair-use-what-transf...
Edit: Note that the other comments saying "I'm just going to wrap an entire operating system in 'AI' to do an end run around copyright" are proposing to do something that wouldn't be transformative and therefore probably wouldn't be fair use. Copyright law has a lot of shades of grey and balancing of factors that make it a lot less "hackable" than those of us who live in the world of code might imagine.
Sure, if I use some code as inspiration for solving a problem at work, that seems fine.
But if I copy verbatim some licensed code then put it in my commercial product, that's the issue.
It's a lot easier to imagine for other applications like generating music. If I trained a music model on publicly available Youtube music videos, then my model generates music identical to Interstellar Love by The Avalanches and I use the "generated" music in my product, that's clearly a use that is against the intent of the law.
It is not uncommon to ask person to "explain in your own words..." - as in use your own abstract internal representation of the learned concepts to demonstrate that you have developed such an abstract internal concept of the topic, and are not merely regurgitating re-disorganized input snippets.
If you don't understand the difference...
edit: That said, if you can create a computer capable of such different abstract thought, congratulations, you've solved the problem of Artificial General Intelligence, and will be welcomed to the Trillionaires' Club
The ability to generalize actually seems to keep increasing with the number of parameters, which is the key interesting result in the GPT-* line of work that Copilot is based on.
Who is in favor of starting it? ;)
https://www.latimes.com/business/la-xpm-2013-jan-17-la-fi-mo...
Correct me if I’m wrong, but is that even possible? I kind of thought that AI is just set of fancy statistical models that requires some (preferably huge) data set in order to infer the best fit. These models can only outperform humans in scenarios where the parameters are well defined.
Many (most?) tasks humans regularly perform don’t have clean and well defined parameters, and there is no AI we can conceive of which are theoretically able to perform the task better then an average human with the adequate training.
It's not possible because of comparative advantage - someone being better than you at literally everything isn't enough to stop you from having a job, because they have better things to do than replace you. Plus "being a human" is a task that people can be employed at.
Why should it be impossible? Arguing that it's impossible for an AI to outperform a human on almost all tasks is like arguing that it's impossible for flying machines to outperform birds.
There's nothing magical going on in our heads. It's just a set of chemical gradients and electrical signals that result in us doing or thinking particular things. Why can't we design a computer that does everything we do... only faster?
You keep using that word. I do not think it means what you think it means.
This does not mean that any GitHub Co-Pilot produced code is suddenly free of license or patent concerns. If the code produces something that matches too closely GPL or otherwise licensed code on a particularly notable algorithm (such as video encoder), you may still be in a difficult legal situation.
You are in essence using "not-your-own-code" by relying on CoPilot, which introduces a risk that the code may not be patent/license free, and you should be aware of the risk if you are using this tool to develop commercial software.
The main issue here is that many average developers may continue to stamp their libraries as MIT/BSD, even though the CoPilot-produced code may not adhere to that license. If the end result is that much of the OSS ecosystem becomes muddied and tainted, this could slowly erode trust in open licenses on GitHub (i.e. the implications would be that open source libraries could become less widely used in commercial applications).
[1] - https://towardsdatascience.com/the-most-important-supreme-co...
This is an excellent example of how the AI singularity/revolution/whatever is a total distraction and that a much bigger and more serious issue is how AI is becoming so effective at turning the output of cheap/free human mental labour into capital. If AI keeps getting better and better and status quo socio-economic structure don't change, trillions in capital will be captured by the 0.01%.
I would be quite a turn up for the books if this AI co-pilot gets suddenly and dramatically better in 2030 and it negatively impacts the software engineering profession. "Hey, that's our code you used to replace us!" we will cry out too late.
I think programming is one of the many domains (including driving) that will never be totally solved by AI unless/until it's full AGI. The long tail of contextual understanding and messy edge-cases is intractable otherwise.
Will that happen one day? Maybe. Will some kinds of labor get fully automated before then? Probably. But I think the overall time-scale is longer than it seems.
Are we in the software community not the ones who have frequently told other industries we have been disrupting to "adapt or die" along with smug remarks about others acting like buggy whip makers? Time to live up to our own words ... if we can.
Now it’s also not necessarily that bad of a state. That’s depending on ensuring a few ground elements are in place like people being able to grow their own food (or supplemental food) or still being free to design and build things on their own. If corporations restrict that then people will be at their mercy for all the essentials of life. My take from history is that I’d prefer to have been a peasant during much of the Middle Ages than a factory worker during the industrial revolution. [1] Then again Chinese people have been willing (seemingly) to leave farms in droves for the last decades to accept the modern version of factory life so perhaps farming peasant life isn’t as idyllic as it’d sound. [2]
1: https://www.lovemoney.com/galleries/84600/how-many-hours-did... 2: https://www.csmonitor.com/2004/0123/p08s01-woap.html
You really want to push for high productivity across all industries, even if that means sacrificing jobs in the short term, because history demonstrated after that, new and more human jobs emerge latter.
How is that different from the current situation?
This is absolutely one of the things that keeps me up at night.
Much of the structure of the modern world hinges on the balance between forces towards consolidation and forces towards fragmentation. We need organizations (by this I mean corporations, governments, unions, etc.) big enough to do big things (like fix climate change) but small enough to not become totalitarian or decrepit.
The forces of consolidation have been winning basically since the 50s with the rise of the military-industrial complex, death of unions, unlimited corporate funding of elections (!), regulatory capture, etc. A short linear extrapolation of the current corporate/government environment in the US is pretty close to Demolition Man's dystopian, "After the franchise wars, all restaurants are Taco Bell."
Big data is a huge force towards consolidation. It's essentially a new form of real estate that can be farmed to grow useful information crops. But it's a strange form of soil that is only productive if you have enough acres of it and whose yield scales superlinearly with the size of your farm.
Imagine doing a self-funded AI startup with just you and a few friends. The idea is nearly unthinkable. How do you bootstrap a data corporation that needs terabytes of information to produce anything of value?
If we don't figure out a "data socialism" movement where people have ownership over the data derived from their life, we will keep careening towards an eventuality where a few giant corporations own the world.
You could call it endgame
And that's why I won't be using it, why give it intelligence so it can work me out of a job?
Umm, no it's not. It's possible we just have two problems - the economic problem you mention might be correct, but also that people who believe in the problems of the singularity are right as well. The existence of a certain problem doesn't negate the existence of the other problem.
I'm curious as to why it seems persuasive. Open source licenses largely hinge on restrictions tied to distribution of the software, and training a model does not constitute as distribution.
1. Programmers will become teachers of the co-pilot through IDE / API feedback
2. Expect CI like services for automated refactoringIn other words, are these systems to be treated like students that learned to perform the task they do from a collection of source material, or are they to be viewed as sophisticated databases that "just" perform context-sensitive retrieval?
These are interesting and important questions and I'm glad someone is publicly asking them and that many of us at least think about them.
It's still amazing to me that (US-centric context here), it's well established that instructions how to turn raw ingredients into a cake are not protectable but code that results in transforming one set of numbers into another are protectable.
AI is just making the silliness of that distinction more obvious.
Why do you think that? A compiler uses human readable code to create machine code, with arbitrary optimizations and choices.
That is really quite debatable in some contexts. Declarative languages like Prolog, SQL, etc. declare what they want and the system figures out how to produce it. Much like a recipe, really.
These reductionist arguments lead nowhere. Fortunately, IP lawyers -- including Microsoft's who are fiercely pro IP when it suits them -- think in a more humanistic way and consider the years of work of the IP creator.
Food recipes are irrelevant; the often go back centuries and it's rather hard to identify individual creators. Not so in software.
That's not correct. Food recipes are created all the time and are attributed. From edible water bottles to impossible burgers, et al.
What is confusing is that the neural net may take lots of small chunks and link them to one another, and then reproduce them in the same order verbatim.
So, the length of the samples being drawn is not necessarily small: the chunk size is based on its commonality. It could easily be long enough to trigger a copyright violation.
Licenses hold no power outside of that granted to it by things being copyrighted by default.
The code that is already used to train should be problematic for them, not only new Code in the future.
At least as long as the system really learns concepts. If it just copy & pastes code, then that's a different story (same as with humans).
It can become a massive (and unfair) competitive advantage.
Furthermore, Copilot will not work with less popular languages and also prevent popular languages from evolving.
I'm impressed. They did an amazing job from a corporate strategy standpoint. Also directionally things are getting interesting
GitHub repositories are open for the taking, GPT-XXX is cloneable (mostly, anyway) and VS Code is extensible.
They definitely have a good head-start, but I really don't think there's anything here that won't be generally available within 2 years.
I took a look at their examples and they are not at all compelling. In one example it generated SQL and somehow knew the columns and tables in a database that it had no context on. So that's a lot of smoke and mirrors going on right there.
Do many developers actually want to work in this manner? That is, being interrupted every time they type with a robot interjection of some Frankenstein code that they now have to go through and review and understand. Personally, this is going to kick me out of the zone/flow too often to be useful. Coding isn't the hard part of my job. If this tool can somehow guess the business requirements of the task at hand, then I'll be impressed.
Even if the tool generates accurate code, if I don't fully understand what it wrote, then what? I'm still stuck digging through documentation and stackoverflow to verify that whatever is in my text editor is correct code. "Code confidently in unfamiliar territory" sounds like a Boeing 737 Max sized disaster in the making.
[1] https://www.plagiarismtoday.com/2008/03/25/iparadigms-wins-t...
Would it be a stretch to assert that GPL'd libraries have a market value for their creator in terms of reputation etc.?
I get it that the derivative work might be more clear in an AI setting, but basically it boils down to the same thing.
"But snippet proposals call out to GH, so they can know which bits of code they generated!". Sometimes; but after Bob does a co-pilot assisted session, and Alice refactors to change a snippet's location and rename some variables and some other minor changes and then commits, can you still tell if it's 95% codex-generated?
musicians, artists, all kinds of athletes, all grow by watching observing and learning from others. as if all these open source projects got to where they are without looking at how others did things.
i don't think a single function, similar syntax or basic check function is worth arguing about, its not like co-pilot is stealing an entire code base and just plopping it out by reading your mind and knowing what you want. i know developers that have certainly stolen code and implementation details from past employers and that was just fine.
We have a guy that brought his task manager codebase (he re-wrote it) but it's the same thing he used at 2 other companies.
I have written 3 MPIs (master person/patient index) at this point all with the same fundamental matching engine.
I mean, one thing we can all agree on is that ML is good at copying what we already do.
But if I do it under a copyleft license like GPL, I expect those who copy to abide by the license and open source their own code too.
But sure, people shit on IP rights all the time, and I am guilty of it too. Let's say I didn't pay what I should have paid for every piece of software I have used.
Now is my produced code is also GPL derivative because I certainly did read through the code base to be able to write larger programs?
"""
"but eevee, humans also learn by reading open source code, so isn't that the same thing" - no - humans are capable of abstract understanding and have a breadth of other knowledge to draw from - statistical models do not - you have fallen for marketing
The real difference is that if one human can learn to code from public sources, then so can anyone else. Nobody is explicitly barred from accessing the same material. The AI, however, is kept proprietary. Nobody else can recreate it because people are explicitly barred from doing so. People cannot access the source code of the training algorithm; people cannot access enough hardware to perform the training; and most people cannot even access the training data. It may consist of repos that are technically all publicly available, but try downloading all of GitHub and see if they let you do that quickly, and/or whether you have enough disk space.
This puts the owners of the AI at a significant advantage over everyone else. I think this is the core of the concern.
This seems pretty backwards to me. A GPL licensed data point is more permissive than an unlicensed data point.
That said, I’m glad that these data points do have explicit licenses that say “if you use this, you must do XYZ” so that it’s clear that our large ML projects are going counter to creators intent when they made it open.
I’d love to start seeing licenses about use as training data. Then maybe we’d see more open access to these models that benefit from the openness of the web. I’d personally use licenses that say if you want to train on my work, you must publish the model. That goes for my code, my writing, and my photography.
Anyways GitHub is arguing that any use of publicly available data for training is fair use, but they also admit that it’s all new and unprecedented, regarding training data.
We should probably start thinking about AI rights at some point. Personally I'll be crediting GPT-3 as any other contributor because it sounds cool but maybe morally too in future
What you can expect a person to do is understand the principles behind that GPL code, and write something along the same lines. GitHub Co-Pilot is not a general ai, and it's not touted as one, so we shouldn't be considering whether it really knows code principles, only that it can reliably output code that fits a similar function to what came before, which could reasonably include entire blocks of GPL code.
It's perfectly fine for me to develop programming skills by reading any code regardless of the license. When a corp snatches an employee from competitors, they get to keep their skills even if they signed an NDA and can't talk about what they worked on. On the other hand there's the no-compete agreement, where you can't. Good luck making a no-compete agreement with a neural network.
Even if someone feeds stolen or illegal data as an input dataset to gain advantage in ML, how do we even prove it if we're only given the trained model and it generalizes well?
> "but eevee, humans also learn by reading open source code, so isn't that the same thing"
> - no
> - humans are capable of abstract understanding and have a breadth of other knowledge to draw from
> - statistical models do not
> - you have fallen for marketing
Compare if you had only learned writing from, say, the Bible. You would probably write in a very Biblical manner, but would you write the Psalms exactly? Most likely not.
All that said, I'm not confident that anyone will stop them in court anyway. This hasn't tenmded to be very easy when companies infringe other open source code copyright terms.
Until it is cleared up though, it would seem extremely unwise for anyone to use any code from it.
You can automate this process by providing existing GPL source code and see what CoPilot comes up next.
I am sure at some point it WILL produce exact the same code snippet from certain GPL project, provided that you have attempted enough times.
Not sure what the legal interpretation would be though, it is pretty gray-ish in that regard.
There would always be risk for CoPilot, had it digested certain PII information and people found it out...it would be much more interesting to see the outcome.
Then a big corporation comes in, appropriates it, repackages and sells as a new product.
It's a shameful behaviour.
And this GitHub co-pilot in no way infringes on full codebases.
Anyway, there is another problem that is patents and is huger, much huger. I think the Apache license has a provision about patents, but most of other licenses may have code that has patents and if the AI generate something similar it may be included in the patent.
If heart surgeons train an AI robot to do heart surgery ... shouldn't they be compensated (as passive income) for enabling that automation?
Shouldn't this all be accounted for? If my code helps you write better code (via AI) shouldn't I be compensated for the value generated?
We are being ripped off.
"Can't wait to see a case for this go in front of an 80 year old judge who rules something arbitrary and justifies it with an inaccurate comparison to something nontechnical."
This implies that by just changing the variable names, the snippets are classed as non-verbatim.
I don't buy that this number is anywhere close to the actual figure if you assume that you can't just change function names and variable names and suddenly say you have escaped both the legality and the spirit of GPL.
To the extent that GPT-3 / co-pilot is just an over-fitted neural net, then it's primary value is as an automated search, copy, and paste.
The top one reads just like an ad: https://news.ycombinator.com/item?id=27676845
Some posts that definitely aren't by shills (including the third one because I simply don't believe there's a person on the planet that "can't remember the last time Windows got in my way"): https://news.ycombinator.com/item?id=27678231 https://news.ycombinator.com/item?id=27686416 https://news.ycombinator.com/item?id=27682270
Very mild, yet negative sentiment opinion (downvoted quickly): https://news.ycombinator.com/item?id=27676942
I don't understand the second sentence, i.e. where's the proof?
Where do you draw the line? That's for the courts to decide!
I'm getting a lot of suggestions that make no sense. What's worse, the suggest code has invalid types, and won't compile. I'm surprised they didn't prune the solution tree via compiler validation.
Reminder - Software engineers, our codes, GPLs are not special.
> "but eevee, humans also learn by reading open source code, so isn't that the same thing" - no - humans are capable of abstract understanding and have a breadth of other knowledge to draw from - statistical models do not - you have fallen for marketing
Machines will draw on other sources of knowledge besides the GPL code. Whether they have the capacity for "abstract thought" is probably up for debate. There's not much else said in those bullets. It's not a good argument.
(Reposting my comment from yesterday)
Interesting copyright issues.
Anyone who thinks their profession will continue as-is for the long term is probably mistaken.
And even once that happens you shouldn't be worried about your job. Why? Because economically everything will be different and because your job isn't that important, it likely never was. The problems humanity faces are existential. Authoritarianism, ecosystem collapse and mass migration of billions of people.
So if you really want to "prepare", then try to make a difference in what actually matters.
Mind-blowingly hilarious armchair criticism.
It's not about progress or supressing it, it's a fundamental question about whether it is OK for huge companies to profit from the work of others without as much as giving credit, and if using AI this way represents an instance of doing so.
The latter aspect goes beyond productivity or licensing - the OP asserts that AI isn't equivalent to a student who learned from examples how to perform a task, but rather replicates (recalls) or reproduces the works of others (e.g. the training material).
It's a question that goes beyond this particular application: what about GAN-based generators? Do they merely reproduce slight variations of the training material? If so, wouldn't the authors of the training material have some kind of intellectual property rights to the generated works?
This doesn't just concern code snippets, it's a general question about AI, crediting creators, and circumventing licensing and intellectual property rights.
We already invented something for that a couple decades ago, and it's called a "library". And unlike this thing, libraries don't launder appropriation of the public commons with total disregard for those who have actually built that commons.
However, legally, the most recent Oracle vs. Google case has already settled a major point: APIs don't violate copyright. And as Github co-pilot is an API (A self-modifying one, but an API nonetheless), Microsoft has a good defense.
In the near-future, when we have AI-assisted reverse engineering along with Github co-pilot, then, with enough obfuscation there's nothing that can't be legally created or recreated on a computer, proprietary or not. This is simultaneously free software's greatest dream and worst nightmare.
Edit: changed Hilary Putnam to John Searle Edit 2: spelling
That's... a mind-bendingly bad take. Google took an API definition and duplicated it; Copilot is taking general code and (allegedly) duplicating it. This was not done in order to enable any sort of interoperability or compatibility.
The "API defense" would apply if Copilot only produced API-related code, or (against CP) if someone reproduced the interfaces copilot exposes to consumers.
> Microsoft has a good defense.
MS has many good defenses (transformative work, github agreements, etc etc), but this is not one of them.
That's a wild misconstrual of what the courts actually ruled in Oracle v. Google.
(And to the reader: don't take cues from people banging out poorly reasoned quasi-legal arguments in off-the-cuff comments.)
I’m just presuming we have a future where you can consume unique content indefinitely. Such as instead of binge watching Star Trek on Netflix you press play and new episodes are generated and played continuously, 24/7, and they are actually really good.
Thus intellectual property becomes a commodity.