Firstly, I will state that I am in full agreement with LeCun on what he states in the video about the need for AI systems such as LLMs to be open [1]. I am not sure if I entirely agree with his very high optimism regarding AI safety, but I have always been far more concerned about current risks and I suspect that in practice we are very much aligned.
[1]: https://yewtu.be/watch?v=EGDG3hgPNp8&t=6525s
Now, let me move to the issue of licensing that I have already talked about at great length in comments on other, older submissions. So let me do something new here and try to put it into a historical context.
Computer science and machine learning in particular has a tradition of openness that goes beyond that of many other fields. What I mean by this is that over the last twenty or so years, there has been a gradual move towards open sharing of research, research outcomes, etc. All under the umbrella of open science and FLOSS. One of the earliest examples I am aware of (there may be others) is how the Journal of Machine Learning Research was established as a direct reaction to the resistance against open access in another leading journal at the time [2]. Similarly, in natural language processing, which my primary field of research, we have an open anthology where the great majority (if not nearly all) of the most important research going all the way back to the 70s is available for everyone to read and use [3]. In AI, we really have pushed back against the big publishers and this is a beautiful thing indeed (one which LeCun has contributed towards at that). So, given the context of this, I believe you can understand the dismay which the field feels when Google Deep Mind publishes papers such as the one on Alpha Go in journals without open access [4]. Yes, getting hold of the papers is not challenging for us, but it clearly goes against the efforts of the great majority of the filed. Thus the dismay.
[2]: https://en.wikipedia.org/wiki/Journal_of_Machine_Learning_Re...
[4]: https://www.nature.com/articles/nature24270
Moving now to FLOSS. When I entered the field nearly 15 years ago, it was somewhat of a rarity to find source code attached to papers. Likewise, training data was also not always made available or came under awful (and expensive) licensing agreements such as those imposed by the Linguistic Data Consortium. However, we as researchers released more and more open data and code, and by 2015 or so it was starting to become the norm that both code and data was attached to papers. Back in those days, training a system given the training data took maybe a day or so, thus model weights were less important. In addition, the models were less complex and thus replicating the training procedure was less problematic. Still, we saw more and more papers being published with their weights. In terms of licenses, there was a healthy mix of MIT-like, Apache 2.0, and GPL. The notable exception in the field would be Google Deep Mind, which despite their prominence at best would publish a library or two. This in contrast to say Google "Proper", which gave us say BERT [5]. It is in this context which I and many others view the last few years of closing down and adding restrictions to the sharing of model code and weights.
[5]: https://aclanthology.org/N19-1423
On the topic of Facebook, I have always considered Facebook AI Research (FAIR) to be the most open of the commercial research labs. For the first few years what FAIR seem to have defaulted to the same open standards as most of the research community. As an example, RoBERTA [6] was shared freely and was the basis for a lot of research for the next few years. However, at some point something changed inside FAIR and instead of releasing model code and weights under permissive licenses, they first changed their default license to exclude commercial usage, and now most recently ship with custom weights-available licenses and usage policies. This is still better than a lot of their competition, but I hope that given the historical context I have provided here you can see why many members of the community see this as the tide turning in terms of openness in the field and we are concerned. It should also be noted that FAIR still uses open science and open source as terms, despite not honouring the history of the terms.
[6]: https://arxiv.org/abs/1907.11692
I want to state that I am not of the position that we are entitled to the work of others. However, I believe the actions of FAIR and others have and will have negative consequences for the long term openness of the field and thus harm both scientific inquiry and industrial innovation. I also believe that it is blatantly dishonest of FAIR and others to claim to support open science and open source, while being in direct violation of this through their actions [7].
[7]: https://blog.opensource.org/metas-llama-2-license-is-not-ope...
A very fair line of reasoning states that what weights-available is still better than what "Open"AI and others are doing. I think this is a good point and I agree with it. Likewise, one could argue that maybe this is the best commercial actors can do in light of the current commercialisation and likely internal political pressure. I again think this is a good point, but I still feel that given the historical context the behaviour is still undesirable. Furthermore, I have heard through colleagues how there is now internal pressure inside several organisations to favour carrying out experiments on their models over those of the competition and I do not believe I need to tell you why this is problematic from a scientific perspective.
To finish this already lengthy post, allow me to highlight some problematic portions of the LLaMa 2 license [8].
> You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof).
This explicitly forbids an ongoing line of research; thus puts limits on the scientific inquiries one can pursue.
> Additional Commercial Terms. If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights.
This is a direct violation of both how the Free Software Foundation and Open Source Initiative defines FLOSS. It also puts up barriers for researchers at private research labs; thus has a chilling effect on the amount of research conducted.
[8]: https://ai.meta.com/llama/license
Apologies for the wall of text. Hopefully some will find some value in my ramblings, just like I have found in yours. I really should get around to writing a weblog...