Besides, it's already pretty unambiguous that weights are not copyrightable: they're a result of a mechanical process. The only original creative input that goes in to the weights is the unfathomable amounts of content scraped from other sources that aren't the authorship of the models. The objective of the gradient descent is simply minimizing loss on the training data.
Facebook doesn't own the llama model weights any more than the Bridgeman Art Library practically owns the paintings of European masters because they made quality scans of them. ( https://en.wikipedia.org/wiki/Bridgeman_Art_Library_v._Corel.... ), or any more than Rural Telephone owns the phone directory ( https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R.... ).
Trying to make model weights copyrightable is going uphill, and I don't see how you get there without first establishing that the these LLM are unlawful derivatives of a countless number of copyrighted works along the way. Doing so would probably create a immediate monopoly for legally created LLMs for the hand full of corporations with quasi-monopoly content hosting services (facebook, google, etc) that can (and/or already have) stuffed licensing into their terms of use.
Do you want a cyberpunk dystopia? I think creating an AI monopoly is how you get a cyberpunk dystopia -- and the two ways we end up with one is either outright restrictions on private development of AI like some have been lobbying for and the other is the extension of copyright so that only a few entities can get access to enough of other people's data at a low enough cost to train them.
I’ll read over your essay and give it some thought. There are a bunch of subtle aspects to consider; I’ve been thinking it over for about four months now and still haven’t covered all the territory yet.
It feels like this may be one of the most important decisions going forward — both from an intellectual property point of view, and an individual rights perspective. E.g. you mention that it’s civil disobedience to share the weights, but it feels like if someone is claiming to do open science (LLaMA), sharing the research materials is the minimum requirement. Plus look how it’s benefited them; they’ve captured most of the open source LLM mindshare. So it seems likely that this will lead to more open source work in the long run, not less.
Feel free to chat! You can DM me on Twitter or email me. I’ve been in the hospital with my wife for 7 weeks, with two to go, so I’ve been a bit less responsive than I usually am.
We will see that anyway. All the code I work on commercially is copyrighted and yet a trade secret. Existence of copyright (with the exception of copyleft, but that's subversion) didn't help software to be open sourced.
IMHO allowing models to be copyrighted is basically 18th century enclosures again.
What do you mean by that? Do you continuously copyright the changes?
Does anybody have a link to a relevant discussion here? I would like to read about the creative process that goes into defining model weights, and how it differs from the mechanical output of running the training algorithm.
Assuming you haven't taken a photo a second for your entire life, then I suspect you'll struggle to make something even close to what's available publically, due to lack of training data.
That said, if you've got a million photos you could probably do some pretty interesting things with very large scale fine tuning, or if you know many other people who have similar stockpiles of photos you may be able to get an entry-level dataset together if you all pool it.