I think we should stop calling this type of models open source. They are indeed "open weight." The training code is proprietary and never revealed.
"Open-source" can be "anything you can go out and grab a copy of and use" but doesn't give you much legal certainty about any of it, and reserve "free software" for the other, better thing.
AGPLv3 partially solves the issue by blocking people like Google from using it to build proprietary cloud services that take away their users' freedom. (It still doesn't solve the problem where providers use network effects to achieve the same end game.)
What in the world do you mean?
On the spectrum of "things that take away user freedom", withholding the source code is bad. Withholding the source code, the binaries and physical access to the computer is obviously much worse! This latter business model is heavily subsidized by GPLv3.
The first sentence of the GNU manifesto says this, and a few sections later in the document elaborate on the point:
https://www.gnu.org/gnu/manifesto.html
Note, in particular, footnote [1] which explains that its OK for distributors to ask for payment, but that it's never OK for users to have to ask for permission to use the software, and the section "Why I Must Write GNU".
Since then, software service monopolies became common, and all of the most end-user-hostile systems on earth rely heavily on the GNU system. At this point, we're paying for permission to use those services with our money, our data, our democracy, etc.
I certainly cannot give you permission to use any of the GPLed services that I have used, or that I've been paid to extend. Therefore, I say the free software movement has lost its way.
As there should be.
I care that I know what I can DO with the project when I see it described as "open source".
Yes, the first of which is that you should be able to build it from source. Which requires the source code, and in this case data.
The problem with requiring "build from scratch" for open source models is that the number of interesting models with training data that can be openly licensed is close to zero.
If you trained your model on an unlicensed scrape of the web you can't release the data under an open source license!
The Open Source Initiative have a bunch of their thinking around this in their FAQ for the "Open Source AI definition": https://opensource.org/ai/faq#isn-t-training-data-required-t...
By this definition almost any binary can be "open source" since hex editors exist. (Or more usefully, you can use ghidra et al. to do more interesting changes.) I know GPL has a very specific view of things, but I'd like to quote an excerpt that I think is generally applicable from https://www.gnu.org/licenses/gpl-3.0.html -
> The “source code” for a work means the preferred form of the work for making modifications to it. “Object code” means any non-source form of a work.
Which is why I'm fine with "open weights", because that's saying the object code is under an open license.
> The problem with requiring "build from scratch" for open source models is that the number of interesting models with training data that can be openly licensed is close to zero.
So? If the number of open source models is zero, then the number of open source models is zero.
It is legal to train on copyrighted materials, provided they were obtained legally. Most companies also train their models using user interactions with previous iterations.
It is impossible to release this data publicly, let alone license it to a third party. However, I believe that at least the training code and the data processing pipeline could, and should, be released in order to claim a model is truly "open source."
That said, Allen AI actually released several models with the full datasets available. It is impressive how they pushed the models' performance despite training on a limited set of publicly available data. Kudos to them.
https://huggingface.co/allenai/OLMo-2-0325-32B
Here's one of their training mixes: https://huggingface.co/datasets/allenai/dolma3_pool - which includes 8 trillion tokens from Common Crawl.
Maybe we should have a little cue card for models: vendor/name, size, open weights, open source, permissive license.
It’s simple enough an idea.
This ship has sailed. It’s now in the same category as hacker/cracker and the pronunciation of GIF.
Some men just want to watch the world burn. At least it's mostly harmless fun anyway. It's even funnier when they bring up how my name is pronounced in defense of "jiff" and I tell them, so you're calling me the expert in "Gi" pronunciation then? :)
Because the other assumption I could have gone with is the less charitable take that they know GIS with a soft G doesn't sound like jizz, but they were just looking for a crude way to mock the soft G.
Way early on (spring 2023) people tried to stop it, but no luck.
A delusion is a false mental belief.
Basically hallucinations are false external things, and delusions false internal things. You hallucinate a pink elephant, you delude yourself into thinking trump won 2020.
I think it's right to push back whenever a huge tech corporation tries to build goodwill by falsely using terms like "open source."
Neither did the inventors of AI. A third party published a document after corporations went with open weights = open source and a spoiler block in FOSS wanted all training data published.
> it's right to push back whenever a huge tech corporation tries to build goodwill by falsely using terms like "open source
I think it’s counterproductive. Most people only see a squabble, which makes any ensuing points from the open-source community seem silly. Those who care can continue using the more-precise language they choose to.
Put another way, there is a difference between using terms like cracker and fully spelling out cryptocurrency, and telling people who use hacker and crypto more loosely that they’re wrong. They aren’t wrong and that isn’t meaningful feedback. At the same time, the person using the precise language isn’t wrong either.
> think it’s counterproductive. Most people only see a squabble, which makes any ensuing points from the open-source community seem silly.
Only to people that truly don't care whether something's open source. In which case, Microsoft using the term (correctly or incorrectly) won't change their perception.
But the people who do care won't like to be mislead by Microsoft. There's a reason the term is right in the headline: people respond to it.
I wish I had time to come up with a better example, but it's like if a AAA game company says they've released "native Linux build," but really they're just packaging the Windows build with Wine.
99% of people won't care, neither about the news nor the deception. But for that last 1%, any goodwill garnered with the headline would be gone, and the game company are the ones who look foolish, not the people calling them out.
Ironically, the roots of the Open Source movement are a direct reponse to the Free Software movement largely because it was considered too ideological and unfriendly to corporate interests (i.e. monetization.)
That said, I entirely agree that MS is misrepresenting their openness here, which isn’t in the least surprising.
[1]: https://opensource.org/licenses [2]: https://opensource.org/osd
Maybe open inference?
But we often also get source code for fine tunning the model.
So maybe it's closer to open source than to anything else?
Isn't it a bit like not calling a game open source because engine tooling used to made it isn't open source and they didn't publish .psd files with asset designs?
As I teach this stuff to people newer to this tech, it's probably just easier and more helpful to refer to the wide array of "stuff you can just download and use yourself" as "open-source" and then after that, go deeper and talk about why Stallman was right, how "Free Software" was first. etc.
Then there's "Smart" in front of Car, Phone, TV, and so on... Meaning different things.
I do think "Open Weight" should be more commonly used. There's definitely communities that spring up that build the training infrastructure and inference infrastructure around open models on the other hand.
> “I think we see a path now where the world gets much more abundant and much better every year.” – Sam Altman