You're giving AI too much credit, it's just a tool, it does not have it's own intentions.
I.e. if you buy a piano or a guitar, you could play and record copyrighted music on it. That's not piano's or guitar's fault though, it's yours.
I.e. if you buy a piano or a guitar, you could play and record copyrighted music on it. That's not piano's or guitar's fault though, it's yours.
Suppose a model was trained solely on a single Beatles album. It could only spit out that album. That would be clear infringement, wouldn't it?
No. It’s not. A Markov chain has some very specific properties that are absolutely not fulfilled by GPT-3 models.
Just say “stochastic” if you want a buzzword. Stop appropriating Markov chains.
What properties does a gpt-3 model have that a Markov chain doesn’t? (Other than effectiveness.)
"Given a prompt, provide a completion" is what a Markov chain does. GPT-3 is exactly the same, in the sense that apples and oranges both satisfy your hunger.
GPT-3 is conditioned on the entire input sequence as well as its own output, which is strictly NON-MARKOVIAN. In fact, the point in saying something is Markovian is exactly that: the state transition probability only depends on the current state.
A Markov chain is a model where: (1) there is a state, (2) the probabilities of the next state depend only on the current state.
That could describe anything from a wet fart to a deterministic computer to GPT-3 to the human mind to quantum mechanics (the real one, not a simulation).
If I unplug the Internet and all USB devices, my computer is a Markov chain with at least several trillion bits of state, so 2^(several trillion) possible states. And there is one next state, which has a probability of 1, and all other possible states have a probability of 0. That's a Markov chain.
GPT models choose the next word probabilistically, with the probabilities chosen by feeding the previous N words into a neural network. That sounds like a Markov chain to me!
This analogy train went too far, don't you think? All examples that I've seen on Twitter require quite an intentional manipulation by human for Copilot to produce something copyrighted. It does not recite Linux code by pressing 1 key.
Surely a judge presented with the "complex series of button pushes," otherwise known as playing an instrument, would hold the player accountable for any infringement and not the piano?
These analogies have gone so far off the rails that I can't tell which side this thread is arguing for by now ;)
One end is GitHub's, at the input: Copilot's "database" was initialized from code that GitHub does not have copyright to. The contention at this end is that they are ignoring the licenses that would grant them the right to use that code.* The article, GitHub, and others assert that there's no copyright issue for creating a database of this kind (a machine learning model).
The other end is the the developer taking Copilot's output. The article seems to take the (absurd IMO) position that there's also no copyright implications here, because the output is not copyrightable at all.
*And personally this is the side that concerns me most.
Edit - googling, the history of player pianos vs copyright is interesting
https://slate.com/technology/2014/05/white-smith-music-case-...
https://www.techdirt.com/articles/20100712/18325210185.shtml
The former for making and selling it, the latter for buying and using it.
Just like counterfeit goods.
No?