I've seen tons of comments here and on Reddit that talk about multiple instances of entire functions being copied verbatim, but the only thing even remotely close to that I've seen is the fast inverse square root, so I must have missed a few tweets or something.
Here's an example from their own docs:
Compare:
https://docs.github.com/assets/images/help/copilot/example_r...
To: https://github.com/nilavghosh/OMSCS/blob/master/AI4Robotics-...
I wouldn't be too surprised either, what surprised me was the large number of people who explicitly said words like "many examples" and then couldn't link more than one.
> Here's an example from their own docs:
Thank you! This is definitely a much stronger example than the Quake one. Looking at the git repository it seems like this is generic startup code for a university assignment, so it probably shows up a significant number of times in the training data.
While it definitely makes sense to interpret a piece of code showing up several hundred times in the exact same format as being okay to straight-up copy+paste (e.g., imports, some boilerplate in more verbose languages, keyboard input switch statements), this does seem to highlight that Copilot can't distinguish between code snippets that are always identical because that's just the correct way to do it, and code snippets that are always identical because only one person did that thing, and it just happens to be in hundreds of repositories for one reason or another.
There's been a lot of bikeshedding on this, but GitHub decidedly hasn't given enough information on how it works and what the training dataset is, and the fair use question definitely needs to be answered, maybe even in court - it's just a matter of time.
All non-private repositories on GitHub.
Pretty much says they did also use other sources.
The first is if GitHub had a license to distribute the code they have learned on. This is assuming that the code it produces is a derivative, which I don't think would be much of a stretch. For this I think most of the licenses that are used, GPL MIT etc, allow for someone to distribute in such a way.
The second is the user of copilot. They would be getting and distributing code where they wouldn't know the original license, but that wouldn't be a viable defense of infringement. To actually comply with most licenses they would have to follow the requirements.
In both cases I don't know if a fair use would really apply. Maybe GitHub could stretch the research aspect, but if you just use the code in your product there is no fair use.
I don't think that's easily settled. Is the code I write a derivative work of all prior code that I've learned from? The difference (IMO) is that Copilot has a much more precise memory than I do; I couldn't recall the exact text of a code example that I've encountered repeatedly even if I wanted to.
If you could it would be copyright infringement anyway.
It also says on their website that the AI may generate api keys that look real but it’s actually just a “fake” key that the AI generated as placeholder.
I have had access for one day, spent my entire Saturday playing with while working on an addon for a game. I find it useful and most of the hints come from other code I have in the same files which saves me time or let’s me know when I’m too repetitive :)
Maybe this is what you meant, but that's already been shown to be untrue. https://fossbytes.com/github-copilot-generating-functional-a...
If you have access, link to the maintainers answer: https://github.com/github/copilot-preview/discussions/45#dis...
The answer references the FAQ which is what I referred to: http://copilot.github.com/#faq-does-github-copilot-ever-outp...
The problem with full snippets of arbitrary obscure repositories being copy+pasted is that there's no realistic way for a user to know if that's happening without putting in more effort than just writing the code themselves, somewhat defeating the point. That's not really case when the first line of the file contains "(C) Someone Else 2003".
You consider a license file to be code?
You said "copilot just regurgitates code snippets", and the example given wasn't a code snippet. Of course it's relevant.
> if copilot is copy+pasting random licenses then it's definitely copy+pasting code as well
Not at all. Copy+pasting licenses is intended behaviour. That's the entire point of having standard licenses. No-one would want Copilot to try to make its own entirely new licenses, therefore it's very trivial to imagine a piece of software designed to allow for licenses to be copied verbatim, but not allow that for any significant chunk of code.
Copilot does not appear to be designed like that, but that doesn't make the logic correct. We only know that it isn't designed like that because we looked for an example of it actually copying arbitrary code.
> I don't see how this could be hard to believe
I don't care about believing it, I care about whether or not it's true. If someone claims to have seen Copilot copy+pasting code, and when asked for evidence their only example is it copy+pasting some license boilerplate, they're just straight-up lying - and worse, potentially drowning out any real examples that exist.
The license/copyright notice (and note that the copyright notice with the names of individuals easily invalidates your assertion regarding boilerplate) issue is that it's clear that no due diligence was done on this product. No one (at least no one with the authority to stop it) spent any time reviewing what this product could produce, because _producing a copyright notice with a random developer's name on it_ is so beyond the pale that even the most cursory review by someone who knew literally anything about the law or copyright would have stopped it in its tracks.
It's not important that the examples are really specific, the issue is that it does happens. This tool has the potential to infringe copyright (sure it does seems legal at some place, doesn't make it more right though).
Look at how EA reversed engineered the Genesis in the past [1]. They had 2 teams, one that did the reversing, and another that did the implementation. That made it safe to say that no infringing code was going through. Plenty of emulators developers try to avoid source code leaks for similar reason. Co-Pilot can't do that.
The fact that the copyright status of the code is hard to determine, doesn't means it's not copyrighted code nonetheless.
Personally I got nothing against this kind of technology, but I do have a pretty big issue with how it learn. Once people published their code on Github, they didn't know it could have been used to do machine learning and it's bad that it is. If Github asked for the copyright over the code to do Co-Pilot, I wouldn't mind.