Llama isn't open source at all. Stop using that phrase for your product featuring even an EULA.
We don't have a commonly-accepted definition of what open source means for LLMs. Just negating Facebook's doesn't advance the discussion.
The open-source community is fractured between those who want it to mean weights available (Facebook); weights and transformer available; weights with no use restrictions (I think this is you); and weights, transformer and training data with no restrictions (obviously not workable, not even the OSI's proposed definition goes that far [1]).
In a world where the only LLMs are proprietary or Llama and the open-source community either remains fractured or chooses an unworkable ask, the latter wil define how the term is used.
[1] https://opensource.org/deepdive/drafts/open-source-ai-defini...
If you distribute the output of bison (say, foo.c) and not the foo.y sources, you would get pushback.
Then there is the EULA which makes it closed source right from the start.
This makes an LLM (emphasis on large) that is open source per this definition legally impossible. In every jurisdiction of consequence.
People like the term open source. It will get used. It is currently undefined. If the choice is an impractical definition and a bad one, we'll get stuck with the bad one. (See: hacker v cracker, crypto(currency) v crypto(graphy), et cetera.)
Maybe. But this isn't how language works. Particularly not English.
A corollary of No true Scotsman [1] is the person administering that purity test rarely gets to define a Scotsman.
When you see a MIT-licensed repository, do you call a screenshot, icons or other image assets "open pictures"? That would be laughable. We don't need non-ambiguity outside of context.
Second, the purism argument is just silly. Is GCC not open source because there's no LaTeX for IA-32 manual, and its x86 machine instruction generator is clearly distilled from it?
Even your reply to this one lacks nuance that I brought up. Where do you draw a line for the things that need "source" vs the ones that don't?
It has nothing to do with natural language evolution.
LLMs were entirely proprietary. In that context, Facebook put forward a foundational model one can run on their own hardware and called it open.
At the time, nobody had defined what an open-source LLM was. People came out to say Llama wasn't open. But nobody rigorously proposed a practical definition. The OSI has started, but they're being honest about it being a draft. In the meantime, people are organically discussing open versus proprietary models in a variety of contexts, most of which aren't particularly concerned with the OSI's definitions.
one is pragmatic, ergonomic, and motivated by advancing relationships between communicating persons in a naturally occurring way because the common denominator is a quick race to mutual understanding.
the other is manufactured, and not motivated by relation and advancing communication, but by how the shift in understanding benefits the one pushing for it, and often involves telling people how to think but is doing it through subversion.
Sort of. I'm claiming the meaning of open source when applied to AI is unsettled. There are guiding principles that seem to imply Llama is not open source. But merely pointing that out without offering a practical alternative definition almost guarantees that the intentionally-misused definition Facebook is promulgating becomes the accepted one.
however i do not think the alternatives need to be proposed at this moment in time because right now, the discussion is about holding people who intentionally reframe and misuse words accountable for their "double-speak" given the term's precedent.
conventionally, and by historical collective understanding, it is not open source.
i get you are attempting to highlight AI perhaps means this should be a definition reconsidered, but the irony here is the message itself is distorted due to the conflation, hence why consistency in language to me seems self-evident as a net good.
there is most certainly a difference between naturally occurring (which our brains reeeally support in terms of language development and symbolic communication), and manufactured (and therefore pushing for a word, or more aptly, a perspective's adoption).
i'd rather words manifest through a common need to reach mutual understanding as a means to relate to one another and this world, rather than having someone who stands to benefit from the change in definition, tell me what it means, and then expect me to just "agree", while they campaign around that and pretend it's the established definition (and not actually their own revised version).
it'd be one thing if people who were throwing the term around so loosely would be transparent: "Hey, we know this isn't historically what everyone means by OSS, but... that's OSS your OSS this is OSS, everything is OSS
instead a lot of these narratives are standing on the shoulders of the original definition and context of what it means to be OSS, and therefore the pedigree, implications (and whatever else for PR spin/influence), and simultaneously diluting what it means in the process as the definition gets further and further obfuscated by those influencing the change, and its pedigree is relied on as a distraction away from what is being done, or actually said.
They could use the more correct Open Weights (which is still a euphemism because of the EULA).
But they do not, and they know perfectly well what they are doing. They are the ones responsible for these discussions, but they double down and blame the true OSS people.
Of course it is. Look at this thread. Look at the policy discussions around regulating AI. Hell, look at the OSI's draft definition [1].
Pretending something is rigorously defined the way you want it to be doesn't make it so.
[1] https://opensource.org/deepdive/drafts/open-source-ai-defini...
"Open-source" represents a cluster of concepts but at the core of it there is a specific definition in spirit at least -- you can see the source for yourself, and compile it for yourself.
If the source is not available, why would you want to call it open-source? Just call it something else. As simple as that.
Definitions flow out of usage. The definition clarifies how the word is used and what people mean when they do use it.
You are, in a very literal sense, doing what Orwell, et al was so desperately against by actively controlling how language is permitted to be used.
Nobody can "force" someone else to use the correct definitions of words, but when people disregard their established meanings they risk communication breaking down and the confusion and misunderstandings that follow. If I went around speaking nonsense or making up my own invented definitions for established words I shouldn't expect to be understood and others would be perfectly right to correct me or ask that I stick to using the well understood and documented meaning of words if I expect to have a productive conversation.
It's also perfectly fair to call out people who twist the meaning of words intentionally so that they can lie, mislead, and manipulate others. When it comes to products, companies can't just say "Words can mean anything I say they do! There are no rules!" to get away with false advertising.
The meaning of words drifts in every living language.
> perfectly fair to call out people who twist the meaning of words intentionally
We don't have consensus around what open source means for LLMs. Facebook is pretending we do. But so is everyone in this thread claiming there is a single true definition of an open source LLM.
And it does result in a lot of confusion and misunderstanding until gradually people are taught the new definitions and how they are used. There are also groups of people who deliberately and continuously redefine words because they don't want to be widely understood. Some want to develop a means to signal to and identify others within their in-group, and some want to keep outsiders from understanding them so they can speak more openly in mixed company.
> We don't have consensus around what open source means for LLMs.
There are people who will argue about what open source means for anything. It's okay that open source means different things to different people, but it does result in confusion in discussions until people make their definitions clear.
I don't think that Facebook has earned the benefit of the doubt, in fact they've more than earned our skepticism, so it's very reasonable to see their new definition of "open source" as being nothing but marketing rhetoric at best, or at worst, as an attempt to twist our still developing consensus on what open source means into something that violates the philosophy/spirit of the open source movement.
Re-definition example: War is peace, freedom is slavery.
Since the new euphemism for downloadable models is a re-definition, Orwell would have been 100% against it. In fact the new use of "open source" is an Orwellian term.
i'm inclined to believe Orwell would have disagreed with OP, and would be asking himself -- why is there such a distinct push by those who benefit from the reframing, to reframe what Open Source means (compared to its already established meaning).
Orwell highlighted and warned against the consequences of people in influential positions of power intentionally distorting the collective associations with their new, updated versions of existing words, campaigning around those distortions, and intentionally reframing associations over time such that, the associations are polarizing and obfuscated, motivated by manipulation to benefit a select few, not motivated by advancing communication -- it certainly was not an example of society and its language naturally evolving "definitions" through usage.
and the novel wasn't a criticism against slang, or association/vernacular changing/evolving over time throughout collective use, nor was it a stance on requiring fixed, permanent, unwavering definitions -- it only emphasized how important it is to have consistent meaning.
he just wanted to encourage people to be skeptical of those pushing for the "different" or updated meaning of words, that clearly had a well-defined context, and association, previously -- why are they so dedicated and determined to "push" for a new meaning to get accepted, when there is a previously established and well accepted meaning already.
that doesn't sound natural to me, that sounds manufactured.
GPLv3 defines "source code" as the preferred form for making changes.
For most normal software that is identical to what you'd use to recreate it... but the way to make changes to an LLM isn't to rebuild it, but is to run fine-tuning on it.
I agree that for Llama, things are weird and they want to cover their bases, and that its better than nothing, but the specific use of “open source” is a long-running corporate dilution of what open source really means and I am tired of it.
The model is a static data file like a word doc.
Meta open sourced the code to run inference on the model (ie the code for Microsoft word reading a doc file).
They also open sourced the code to train/fine tune the model. (Ie the code for Microsoft word writing a doc file)
Then they released a special doc (the llama 3 model), but didn’t include the script they used to create that doc.
Anything else and you're just open-source-washing your proprietary technology.
Llama did not license its training data. It’s almost impossible to prove a particular LLM was used to generate any particular text, and there’s likely a bunch of illegal content within the dataset used to train the model (as is the case for most other LLMs)…
So why should I care about following a license? They have no mechanism to enforce it. They have no mechanism to detect when it’s being violated. They themselves indicated hostilities to other licenses, so why not ignore it?
i.e. high temperature, exotic samplers (i.e. typicality sampling), using lora/soft prompts/representation engineering, using it as part of a chain on top of other AI, etc
I don't care if every lawyer on earth is hired by Meta. Show me evidence that any particular LLM can be trivially fingerprinted based on its outputs. No, that "red token green token" paper on watermarking (https://arxiv.org/pdf/2301.10226) is not an example of this because of how trivial it is to defeat.
Edit: I can't reply to the comment saying "Subpoena" but this commentator seems to think that using any LLM at all is grounds for a court to issue a Subpoena requiring you to disclose which LLM you're using. If this actually happened, you'd see a massive chilling effect. Also, what stops someone from silently replacing the model with a non infringing one the moment someone starts asking questions?
I'm pretty sure that most courts aren't capable of getting expert testimony which is good enough to deduce that I silently swapped out my blarg_3.1 model which was made using a 1/3 llama3 merge with something else with gloop_1.5 which is no longer infringing.
Like seriously, I again ask, given the idea of courts with warrants and Subpoena's, why should I care about meta's licensing?
Edit2: If you're afraid of an employee leaking this info, don't tell your employees. Good thing clever model merging leaves no traces if you delete metadata!
Subpoenas.
https://www.forbes.com/sites/alexandralevine/2023/12/20/stab...
https://www.airforcetimes.com/news/your-air-force/2024/03/26... (i.e. as source of where LLMs get their classified data from)
If you scrape a large enough part of the internet, you're naturally going to get extremely illegal training data that you won't effectively filter out. I guarantee you that at least a tiny bit of highly classified information was not filtered out of most LLM training data (it wasn't found during the filter step), and it's quite remarkable that this and the above revelations have not led to anyone being Subpoena'd or related in regards to it.
So no, I think that folks will be literally the opposite of litigious on this issue. You want to play that game Zuck? Let's see what happens when I hire my researchers to find the dirt on your models dataset. We will see then who "settles out of court".
Meanwhile, all those lawyers need is to grab your text messages, emails, and slack messages. There's no need to look at code. If you have been up to something it's likely to come out pretty quickly.
p.s., Hiding your tracks after a subpoena is issued is a good way to end up in jail, at least in the US.
Once again: for those who are new here. There is Free Software, which has a usefully strict definition.
And there is Open Source, the business-friendly -- but consequently looser -- other thing.
You can like one or both, they both have advantages and drawbacks.
But you cannot insist that "Open Source" has a very strict definition. It just doesn't. That's why the whole Free Software thing is needed, and IMHO, more important.
Statements like this remind me that I really need to KEEP GOING with this.
One could argue that the OSI should have gotten a trademark on the term. But the FSF doesn't have a trademark on the term "free Software" either, so the terms have approximately equal legal protections.
Meta using the term "open source" to apply to their model data when their license isn't an open source license is dishonest at best.
Free Software has the GPL, and all its related, healthy controversy. It's not perfectly clear, but it's far more battle-tested than the much more nebulous "Open source."
People who like "free software" put in work, and better understood that, to some extent, you can't have your cake and eat it too. "Open Source" is much more about a whole lot (to me, naive) wishful thinking.
(The OSI is a bunch of companies in a trenchcoat, the creators of the GPL were more principled.)
I just noticed they are currently discussing what "Open Source AI" should mean. You can join in and add your thoughts tot he discussion.