Maybe by bureaucrats in the OSI. Meta through the Llama models have done more for open source LLMs than just about anyone else, which the community recognises.
The perfect is the enemy of the good, as usual.
The perfect is the enemy of the good, as usual.
The whole crux is to get an objective definition of what "open AI" is, and how it needs to behave. If others agree (or disagree) with the vendor that doesn't mean the vendor is permitted to make its unilateral de facto declaration a de jure consensus decision.
That's not how standards work.
Is it supposed to give the false impression that OpenAI and Meta’s actions have been equivalent with respect to releasing their models? And if it isn’t an attempt to mislead in such a way, then what exactly does it mean?
I guess they’re trying to call out OpenAI more than group them with Meta
Yes, some data is open and the creators of it want it to be shared openly.
Other people want their copyrights upheld and for there to be no intellectual property appropriation (theft) of their creations.
There is also the issue of aggregation. For example, Wikipedia is an open and free aggregation of knowledge. But it also cites, quotes, and links out to sources which may be copyrighted works.
Note that Wikipedia has very formal guidance as to how to avoid and handle instances of plagiarism.
https://en.wikipedia.org/wiki/Wikipedia:Plagiarism
AI needs to face the fact that it is rife with intellectual property theft. It needs standards for infringed copyright removal. It needs standards for the defense of trademarks. It needs privacy standards to prevent the unauthorized use of personal data, images, recordings and other works from people around the globe.
If AI is permitted to simply glut itself on every artistic creation and every human on the planet, sans limitations, it is basically the equivalent of a crime spree.
So, the choice is give up everything that gives you any advantage, and hence any incentive to develop anything in the first place, or you’re the bad guy.
I’m not sure how they think the world works. But I agree that we should be clear about our standards. So I’d suggest that those with these expectations adopt a new brand other than “open source”. Maybe “puritan approved”.
Except it's Mark and Meta, which are household names, and I'm some Internet rando, so my voice is smaller, hence the accusations of bullying. If someone decides for you that your new name is shithead, and everyone goes along with it because that person's bigger than you, thats bullying.
That’s really the equivalent of what’s happening here?
In this particular case, meta's models are not "open source" although Zucks calls them as such. And because of that, people also started to believe they are open source.
Open Source AI will be able to reference and share its underlying truth data. Imagine an Open Source AI trained on all GPL code or a Public Library and it could guide you directly to the middle of the source of any code / book it knows.
"But not everyone is going to spend the money or time to train their own models from scratch!" I hear you say. And there's some truth to that. But if we had truly open source LLMs, then AuroraGPT development would be fast tracked and we would have a fully government funded scientific frontier model instead of only fine-tuning models.
This intention necessarily translates to models. In order to reproduce a program, the source code and a relevant compiler is what is required. In order to reproduce a model, the architecture (source code), training data and initial parameters are required.
With code, you just compile and run it. Models require carefully curated training data (lots of it), training code, final weights, production inference code, etc.
The danger is the world gets drunk on Meta's models, but then Meta pulls the plug. Meta is doing the world a lot of good with Llama, but we'll be left in a precarious position should they stop being so generous. The next generation of LLMs might make Llama completely pointless.
Meta thinks a great deal about the strategy in this space. They license other weights and models as CC-BY-NC and other non-free licenses because they know that they have the SOTA models across several categories and that there's no other competition in the marketplace. They want to retain their advantage when it benefits them.
So what happens when the LLM space shifts and Meta no longer needs to appear to be open source? What if the value switches over to diffusion models and away from LLMs, which is an area where Meta shines?
Llama is a gift we can't replicate or properly repair and extend. And we need to be careful.
The point is cultivating dependency under the color of "open source" is not generally wise.
Users of the "open source" LLM do not have all the software freedoms normally associated with "open source."
They cannot rebuild it and more. Important pieces are missing the "open" part.
Someone says, "here, take this awesome math library, it is open." Then users find out there is a brittle binary blob needed for the whole thing to work.
Nobody will want to build on or with it because that blob works, until it doesn't and when it does work, people are not sure what it does exactly.
Makes the whole thing a lot more like free beer.
In the scheme of things right now, particularly given both the pace of change, and up coming corruption due to feeding models their own output, anyone's latest may just not be relevant that long.
The fact that llama is not open source does not make it less useful or less appreciated. Nor should anything be considered open source because it is useful or appreciated