Why did Meta open-source Llama 2?
matt-rickard.com
matt-rickard.com
Open Source has a very clear definition. Llama 2 fails in multiple regards.
First, the license by itself is not an open source license. It has important restrictions that make it non-open source.
Second, the distribution. You have to apply for the download with a web form and you are not allowed to redistribute the model.
Third, the source code, i.e. the data used to train the model. Meta is not telling us about it. One of the core features of open source is that you can recreate the binary (in this case, the model weights) by yourself. You can't do that here.
A model is not a binary. Training code is not source code for the model. The training code is closer to an editor. You can release open source software created by a closed source IDE.
you cannot generate the model without the same code and training data.
it is not open source.
you are conflating editing the model with the capability to edit a thing. using your source code example, altering source code changes the binary. as if one were editing the binary.
the source code isn’t called an IDE.
That's great.
I disagree about the other part.
Personally I feel like it's a dangerous precedent because it shifts open source from a community concept to something that a company lets you have with a bunch of conditions.
There are new models/papers/backends coming out all the time (including SDXL, which is similar to the original SD), but the community is so entrenched in Stable Diffusion 1.5 and the old Stability AI PyTorch implementation that moving to anything else hasn't really happened yet.
That being said, I think Llama won't stick around as the "de facto" standard for over a year unless Meta keeps releasing better foundational models.
It's not just inertia, the later models (even 1.5 over 1.4) are heavily censored. And at least in the case of 1.5 it appears to cause obviously inferior output even for content which is in no way explicit.
My understanding is the facebook published Llama 2 chat fine tunes are more or less the most censored LLM yet, refusing to even discuss negative sentiments. I bet we'll see other people's RLHF fine tunes on the base model being used as a point for more developent.
It's an unfortunate development that it appears that the primary commercial "value add" is building models that don't do what the users unambiguously direct them to do. It's hard to make a model smarter, but easier to lobotomize one.
1.5 is better than 1.4, but 2.0/2.1 are indeed heavily lobotomized.
The jury is still out on SDXL, but I think its much better. But the second thing I alluded to, the backend, is going to be more of an issue.
The chat finetunes of Llama 2 are kinda irrelevant because the community will come up with their own very uncensored finetunes.
Which they never apologized for!
Because you think that Protobuf, Android, Chromium, and all the other open source projects by Big Tech are there for the community? It's about control.
Not saying it's fundamentally bad, but you can't deny that the fact that Android exists means that it is very difficult to get adoption for an alternative "open" mobile OS, and Google benefits greatly from it. Same for Chromium, and basically everything.
When Big Tech open sources something, it's a strategic decision. Not philanthropy.
So from Google's perspective, not open sourcing Protobuf does not really give them an edge: alternatives exist. However, by open sourcing it, they end up with a ton of devs who learn Protobuf outside of Google. That makes it easier for Google technology to be adopted if they release something new, and when they hire a new engineer, it's a win if he already knows Protobuf.
Same applies to gRPC and others.
It's not really a unique precedent, Chromium and Android fit the same model, but it's still very much better than closed-source (electron and custom Android ROMs are something you couldn't have otherwise).
> You don’t get the source
You do IIUC. But your can't use it for all purposes (like serving 1B users), which is what makes it non open source.
> v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof).
> 2. Additional Commercial Terms. If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights.
> Prohibited Uses: <a whole page full of text>
LLaMA2 isn't "Open Source" - and why it doesn't matter https://www.alessiofanelli.com/blog/llama2-isnt-open-source
Meta can call Llama open source as much as it likes, but that doesn't mean it is https://www.theregister.com/2023/07/21/llama_is_not_open_sou...
Software licenses masquerading as open source (I wrote that one) http://marble.onl/posts/software-licenses-masquerading-as-op...
There have been lots of other posts on here about this too.
This means that the term "open source" is not legally defined (and not recognized by lawyer / judges) [and the OSD], right?
--- " If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion. --
Doing so, technically, takes the license out of the category of “Open Source.”
Sources: Open Source Initiate Argument https://blog.opensource.org/metas-llama-2-license-is-not-ope...
Llama Terms https://ai.meta.com/resources/models-and-libraries/llama-dow...
This obviously means that the weights were released, and everyone knows this, regardless of any pedantic definition of what "open source" means.
I keep seeing this, but I can't understand how it would work.
Bored nerds will volunteer their time to open source, with little return. That's why I did it.
Generally, nobody will volunteer their money to open source. This is unfortunate, and a huge problem.
Training neural nets requires the money. Where does it come from?
Plus it can be a strategic choice for institutions and companies, just like Linux is for example.
it's not pedantry unless it's a minor detail, and given that the entire license behind llama has next to nothing to do with the open source model or existing open source licensing, this isn't a minor detail.
the court-room doesn't care if 'few people care'.
it's obvious to anyone that has been to a CS convention in the past 20 years that the whole open-source thing from large corporations is used to leverage good-will to recruit useful labor and to collect community 'altruism-points' so that they can make shadier decisions later on and hopefully trap a captive audience, not to follow the ethical ideals of open-source but to enhance their bottom-line.
'Trojan-horse open-source'.
Not sure if you were aware, but nobody is in court right now.
>it's not pedantry unless it's a minor detail
It is minor though. The license allows almost everyone to use it, excluding literally like Apple and Google, and for a few other unenforceable use cases.
Everyone else is in the clear.
It's no surprise you're disinterested in the meaning of "open source" given how much you misuse "everyone".
But feel free to reread the original comment and actually talk about the substance of the claim, instead of doing the normal "Well actually" hacker news comment.
I'd rather say: everyone who actually checked the license knows that it's obviously not open source in any way.
Which is the point. No need to play dumb. You are not confused about what was released.
So great! You are not confused at all, and are aware that the announcement meant that the weights were released!
See! Actual usage! The sky is now green.
The only reason people say that Llama 2 is open source is because they have been misled, not because the definition is changing. There's a lot of goodwill around the term "open source" because of its actual meaning, so it will always be tempting to abuse it for your own benefit, as not everyone bothers to check whether your claims are actually true.
Yep. We don't need additional confusion on this term.
[1] https://www.qualcomm.com/news/releases/2023/07/qualcomm-work...