The weights aren't created by humans so they aren't copyrightable - at least in theory.
Software wasn't for sure copyrightable before 1976 in the US, but there was plenty of closed source code that simply only ever distributed the binaries.
This right isn't protected by copyright, it's protected by the lack of copyright, and how executables are directly modifiable. So the AGPL won't be possible, but it's a lot like permissive licenses, without attribution.
Also, the license to the model assumes copyright (Apache 2) and works by granting you permissions based on that copyright.
A set of weights is a pre-trained local minima in the model space is a both executable and modifiable. It's usually far more useful than the source training data, because the work has been done.
Tons of binary patching works that way. For instance from the gameshark days it was relatively common for a patch that worked for unknown reasons, but who's discovery was tooling assisted and simply displayed a desired effect (and commonly a lot of undesired effects that weren't clear).
> A set of weights is a pre-trained local minima in the model space is a both executable and modifiable.
None of that changes whether it's open source or not. I guarantee you that mistral has some code (and a lot o data) laying around that created this model.
> It's usually far more useful than the source training data, because the work has been done.
For certain operations maybe. I guarantee that Mistral wouldn't constrain themselves to fine-tuning if they had a major change to make to the model.
With these large models, the weights really are quite useful objects in themselves, the very starting points of future modifications, and importantly the desired points of modifications
Practioners in the field call these models "open source" and write open source software to run them and use them to power open source software.
> With these large models, the weights really are quite useful objects in themselves, the very starting points of future modifications, and importantly the desired points of modifications
For fine tuning. You go back to the training pipeline to make major changes.
> Practioners in the field call these models "open source"
Some do. In contrast to the accepted definitions of open source.
> and write open source software to run them and use them to power open source software.
There's plenty of open source code to run closed source binaries.
Obviously zipping something is not enough, even lossy compression is (obviously) not enough. But then how does a language model differ from a lossy compression? Is it just the compression ratio? (are the weights even that much smaller than the data?)
There are ways to train models that guarantee that the amount of information transferred per data point is limited, but to my knowledge those aren't used (and may be prohibitively expensive).
At the core the question is how much artistic input there is in creating LLM models. Are the choice of the model architecture, hyperparameters and training data artistic choices comparable to those by a photographer setting up a shot? Or are they more comparable to technical work that's only protectable by trade secrets, patents and trademarks?
In the same way, data in a database are usually not copyrightable. They are still the company's property (excluding issues with PII and such).
Also, people go to jail for leaking trade secrets, and this certainly qualifies.
Their weights obviously aren't protected by trademark. So, what IP regime protects the weights?
While there is a discussion to be had on sovereignty and international reaches of local laws (such as DMCA for instance), I think it's disingenuous to consider only US legal point of view.
[1] https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CEL... [2] https://www.legifrance.gouv.fr/loda/id/JORFTEXT000000573438/...
No US court is going to invent new rights under US law because a company happens to be incorporated in a foreign jurisdiction.
> While there is a discussion to be had on sovereignty and international reaches of local laws (such as DMCA for instance)
The DMCA (and GDPR) do not have international reach. The DMCA only appears to apply internationally because it binds a whole series of American companies with an international presence. Meanwhile the GDPR protects European residents. A company dealing with these residents has crossed the border, so to speak.
> I think it's disingenuous to consider only US legal point of view.
Candidly, you're either confused about what disingenuous means or you're being disingenuous yourself.
And by the fact that DMCA Takedown Notice and processes are valid in many countries. You can send a valid takedown to a french company and they'll take the content down. Not because of DMCA law in the US or because of American companies, but because the notice is accepted and has legal value within the scope of the local law.
> Their weights obviously aren't protected by trademark. So, what IP regime protects the weights?
> No US court is going to invent new rights under US law because a company happens to be incorporated in a foreign jurisdiction.
Your original question doesn't specify "in the US". A French company wouldn't be able to copy these without issues.
Does that mean you need to be protected everywhere to be qualified as protected? Or simply in the US? If so, then you can use your own argument to say that a Chinese court (or Marshall Islands I guess?) won't create a law for a company incorporated in the US. Which in turn means that IP is not protecting anything since it doesn't apply to every single country on the world with no exceptions.
And yet, IP does protect things. You can't both not scope your question and except a reponse scoped only to the US.