Hacker News Reader

Comment by belter | Hacker News Reader

One interesting aspect, that I thing will make it difficult for GitHub to argue and justify its not a a license violation would be the answer to the following question: Was Copilot trained using Microsoft internal source code or will it be in the future ?

As GitHub is a Microsoft company and OpenAI although a non-profit just got a massive one billion investment from Microsoft (presumably not for free), will it start spitting out once in a while Windows kernel code ? :-)

And if it was NOT trained on Microsoft source code, because it could starting suggesting some of it...Is that not a validation that the results it produces are a derivative work based on the work of the open source code corpus it was trained on ? IANAL...

View on HN

Alternatively, wait for co-pilot to add support for C++, then start writing an operating system with Win32-compatible API using co-pilot.

There is plenty of leaked Windows source code on Github, so chances are that co-pilot would give quite good suggestions for implementing a Win32-compatible kernel. Then watch and see if Microsoft will try to argue that you are violating their copyright using code generated by their AI.

Oh man, that got meta super fast. Its like a mobius strip!

It can always get more meta.

For example, the AI tool that Microsoft's lawyers use ("Co-Counsel"), will be filing the DMCA notices and subsequenct lawsuits against Co-Pilot generated code.

This will result in a massive caseload for the courts, so naturally they'll turn to their AI tool ("DocketPlus Pro") to adjudicate all the cases.

Only thing left is to enter these AI-generated judgements into Etherium smart contracts. Then it's just computers suing other computers, and being ordered to send the fruits of their hashing to one another.

And while the machines are distracted by all that, we can get back to writing code.

Who could have predicted machines would be very good at multitasking. As of today they are STIL writing code AND creating more wealth through gold hoarding AND smart contracts at the same time!

Don't forget settlements paid in Ai-generated crypto-currencies backed by Gold mined in Australia fully automated mine. Run it all on solar and humans can just fuck right off.

Nick Land-style accelerationism, or the "ascended economy". https://slatestarcodex.com/2016/05/30/ascended-economy/

The market ultimately obeys customer demand, so all these problems will be sorted out... until customer AI.

That's the next step for Amazon Prime: 0-click shopping. Just buys stuff from your recommendations every month and sends it to you.

"Local man in mental distress as microwave-lasagna meals get delivered to his house every 20 minutes"

Amazon refuses request for microwave.

The legal system moves swiftly now that we've abolished all lawyers!

Have you read Accelerando by 'cstross? It plays out kind of like this, only taken to a tangent. Notably, it's written before ethereum or bitcoin were conceived. Great storyline.

https://en.wikipedia.org/wiki/Accelerando

I have not. But I will. Thanks!

Isn't this similar to how ads and adblocker fight, just extrapolated?

Yes.

Somehow, I find this a plausible and not entirely undesirable outcome for society. The less time humans spend interfacing with machines, the more points humanity gets anyway.

The nice thing about co-pilot is that it will suggest to do the same mistakes as in other software. If you accept all autosuggestions in C++ you might end up with Windows.

And eventually you will be forced to do it the way everyone does it.

An imaginary conversation between a real developer and some kind of managing person:

"Why are you typing all this stuff by hand? All your coworkers are much more efficient by using the AI!"

"But I need to actually understand ..."

"You should get more efficient! Look at how much time this costs us."

"Yeah but they are copying in mistakes from ..."

"No, the system works! Just do it like everyone else does it and do not waste more time!"

Or at the next code interview ...

This is such a ridiculous statement to me. If this were a real problem we would have noticed by now with stackoverflow. I truly believe the vast majority of capable developers read, understand and test code they copy from somewhere. This is even more obvious with an AI that will never suggest 100% correct code all the time.

oracle is probably already arming their lawyers. just setup a git, put a restrictive license, and scan any new github projects.

Without weighing in on the overall question of “is this a license violation”, you’ve created a false dichotomy.

“GitHub included Microsoft proprietary code in the training set because they view the results as non-derivative” and “GitHub didn’t include Microsoft proprietary code because they view the results as derivative” are clearly not the only options. They could have not included Microsoft internal code because it was way easier to just use the entire open source corpus, for example.

Or: they used the entire open source corpus because they thought it was free for the taking, and when people point out that it is not (that there are licenses) they spin that (claim that only 0.1% of output is directly copied, but that would mean 100 lines in 100k program) and pass any risk onto the user (saying it is their responsibility to vet any code they produce). So they aren't saying that users are in the clear, just that it isn't their problem.

Use neural indexes to find the code that most closely matches the output. Explainable AI should be able to tell you where the autocompletion results came from, even if it is a weighted set of files.

That's a good idea in theory, but the smarter the agent gets, the less direct the derivation and the harder to explain it (and to check the explanation). We're already a long way from a nearest-neighbor model.

Yet the equivalent problem for humans gets addressed by the clean-room approach. This seems unfair.

> the smarter the agent gets, the less direct the derivation and the harder to explain it

at some point it should be different enough to stand on its own, right? then we have no problem with copyrights

Yeah, also in principle. But the cleanroom approach isn't technically required for humans either -- it became standard because the legal notion of a derived work is very fuzzy and gradually changing, and lawsuits are expensive and chancy, so you want a process that's provably not infringing. "Yeah I learned some general ideas from this code, but I didn't derive any of my code from theirs" seems to be a logical rats-nest. With the explainable-AI approach to this particular problem, the more intelligent the AI, the more this solution is like analyzing brain scans of your engineers. If your engineers could have produced "derived work" without literal copying, why can't the AI?

I agree, but we aren't anywhere close to that level yet. For that to be true, I think the AI should possibly have the ability to explain the code it created. What we have now is basically a fancy markov adlib code completion tool.

A more intelligent agent should be able to tell you where it learned all of its knowledge from. I personally would like my AI to be above "gut level instincts" otherwise it reinforces blind trust.

> 100 lines in 100k program

The intention is autocomplete boilerplate, not write a kernel.

This is not a difference in kind.

Autocomplete, do you have anything to say to the commenter ?

“This isn’t the best thing to say.”

Coding a snippet is not different in kind from designing a Kernel? It's the difference between tactics and strategy.

I’ll direct you to my other comment in this thread. But give you the TL;DNR no it isn’t.

How is designing a very large system even close to the same thing as writing a few small functions? That's like saying an architect designing a building is doing the same thing as a brick layer putting down cement.

“””Computers can already author documents at near human quality. Research is continuing to increase the accuracy and volume of these models.

Language processing research will not only help doctors, but will allow machine-based language translation, and eventually automated chat bots that can converse in our languages.

The next steps in human-machine collaboration are to allow people and machines to co-create. A recent Chinese report suggests that 50% of scientific papers in this field will be written without human intervention by 2033, compared with only 11% today.

One of the biggest challenges of machine learning is giving the machine what it lacks. This usually means gaining enough training data to teach the algorithm how to make inferences from data points it has never encountered before.

Many of the large organisations involved in advancing AI's ability to develop documents can improve how the algorithms learn by building on the knowledge and experience of human workers.”””

The above text was automatically written by https://app.inferkit.com/demo . It uses a language model to predict the next word in a sequence. In other words, to use your example, it not only architects, but builds, the entire building simply by predicting where to put the next brick.

So to answer your question: Yes. That’s exactly how it’s done.

And such a thing has never been achieved with code. Besides very often the texts such an ai creates are non-sensical. And they are very short. Writing a few pages of text would equivalent to small tool of a few hundred lines. Or about the same as building a wooden shed. You don't need much skill for that. Come back when an AI can write multiple internally consistent books such as LOTR and the Dilation or the Harry Potter series. That's the scale of architecting a system.

True, but I also think this is showing a lack of imagination about where things are going.

You're trying to say architecting is some big woo idea that's somehow different from writing code. Kind of, maybe. But I bet you could build a functional kernel with central design. Given that's how biological systems work, I'm sure it could be done. Then what say you?

Also, 0.1% of output is directly copied doesn't include the lines where the variable names were slightly changed, but the code was still copied.

If you got the Microsoft codebase and Ctrl+F'd all the variable names and renamed them, I bet they would still argue that the compiled program was still a copy.

>saying it is their responsibility to vet any code they produce

But, if some of the code produced is covered by copyright, isn't Microsoft in trouble for distributing software that distributes copyrighted code without a license? How would it be different from giving out bootlegs DVDs and trying to avoid blame by reminding everyone that the recipients don't own the copyright?

this complicated copyright problem shows we're still using last century concepts on new and emerging technology that surpassed it; it's time to think hard about it because we need neural nets and they need training data

Some are more equal than others though, aren't they? I mean, if MS throws out licensed code from others, as if to say: "Ahh, software licensing, such an outdated concept ..." but then keeps its own code out of that loop. "Yeah, but that's our own code, no one is allowed to copy that!"

I doubt they will corner the market for AI code assistants. ML models are replicated or surpassed in a few months by the competition. We will all benefit from them, it won't remain concentrated in a few hands.

> They could have not included Microsoft internal code because it was way easier to just use the entire open source corpus, for example.

They don't claim they used an “open source corpus” but “public code” because such use is “fair use” not subject to the exclusive rights under copyright.

They could have just used their own open source code, of which they now have plenty, in many languages.

> One interesting aspect, that I thing will make it difficult for GitHub to argue and justify its not a a license violation

They don't claim it wouldn't be a license violation, they claim licensing is irrelevant because copyright protection doesn't apply.

> And if it was NOT trained on Microsoft source code, because it could starting suggesting some of it...Is that not a validation that the results it produces are a derivative work based on the work of the open source code corpus it was trained on ?

No, that would just show them to not want to expose their proprietary code. It doesn't prove anything about derivative works.

Also, their own claim is not that the results aren't a derivative work but that training an AI is fair use, which is an exception to the exclusive rights under copyright, including the exclusive right to create derivative works.

Late to the thread but: OpenAI is not a non-profit since 2019 (technically they call it a capped profit company [1], but until the singularity you can ignore the cap). I guess this does impact the dynamic with Microsoft

[1] https://openai.com/blog/openai-lp/

It probably wasn't because Github is treated as a separate company by Microsoft.

Literally people need to quit Microsoft and join Github to take a role at Github.

That's an interesting employment detail, but what does it have to do with the other parts of the organization? I happen to know that they work together on security and contract areas, and it wouldn't surprise me if there were other similar arrangements in place.

>> Was Copilot trained using Microsoft internal source code...

They explicitly state "public" code so the answer is most certainly "no".

Well, the source of various Windows versions is public on GitHub ...

Not a problem because it's possible to check if the code is verbatim from the training set (bloom filters).

It's not clear to me that verbatim would be the only issue. It might produce lines that are similar, but not identical.

The underlying question is whether the output is a derivative work of the training set? Sidestepping similar issues is why GCC and LLVM have compiler exemptions in their respective licenses.

If simple snippet similarity is enough to trigger the GPL copyright defense I think it goes too far. Seems like GPL has become an obstacle to invention. I learned to run away when I see it.

If that's the case then GPL code should not have been used in the training set. Open AI should have learned to run away when they saw it. The GPL is purposely designed to protect user freedom (it does not care about any special developer freedom) which is it's biggest advantage.

It's not limited to similar or identical code. The issue applies to anything 'derived' from copyrighted code. The issue is simply most visible with similar or identical code.

If you have code from an independent origin, this issue doesn't apply. That's how clean room designs bypass copyright. Similarly if the upstream code waives its copyright in certain types of derived works (compiler/runtime exemptions), it doesn't apply.

So if you work on an open source project and learn some techniques from it, and then in your day job you use a similar technique, is that a copyright violation?

Basically does reading GPL code pollute your brain and make it impossible to work for pay later?

If so you should only ever read BSD code, not GPL.

> Basically does reading GPL code pollute your brain and make it impossible to work for pay later?

It seems to me that some people believe it does. Some of the "clean room" projects specifically instructed developers to not even look at GPL code. Specific examples not at hand.

I start seeing Ballmer's point of view. It's like cancer.

Microsoft appears to believe this (or maybe just MacBU) because I've met employees who tell me they're not allowed to read any public code including Stack Overflow answers.

This has nothing to do with GPL. Copyright is copyright. You can’t even count on public domain everywhere in the world.

Don't come in here with your common sense

Since quite a lot of Microsoft code is on GitHub, I'd say yes.

The "because" in your last bit is a huge leap.

It wasn't trained on internal Microsoft code because the training set is publicly available code. It has nothing to do with whether or not it suggests exactly identical, functionally identical, or similar code. MS internal isn't publicly available. Copilot is trained on publicly available code.

You stated a fact "Copilot is trained on publicly available code".

The question (and implication) is: why not train it on MS internal code, if the claim that the output isn't license-incompatible is true.

If the output doesn't conflict with any open-source license (ie. it springs into existence from general principles, not from "copying" licensed code -- then MS-internal (in fact, any closed-source code) should be open-season.

I can imagine a few of the non-obvious segments of code I've written being "recognizable" methods to solve certain problems. And, they are certainly licensed (GPL + Commercial, in my case).

I think, at the very least, that a set of AIs should be trained on different compatible sets of code, eg. GPL, AGPL, BSD, etc. Then, you could select what amount of license-overlap is compatible with your project.

Reply on news.ycombinator.com