I've heard rumors that it had to do with talent loss, but just rumors.
The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you.
Do weights provide either of those qualities?
> Do weights provide either of those qualities?
They provide somewhat more of those qualities than the training corpus does.
Not a lot, especially for "understanding", but more.
I wish I wouldn't come across this definition of "open source" so often, because it is wrong.
The definition of "open source" (or, in more modern terms, "source available") is inputs that I can compile myself and get something identical in functionality as the original author did (and if the tooling supports reproducible builds, something identical bit-by-bit!).
An "open source" ML model is not fulfilling that definition - it is only compiled output, similar to a piece of proprietary software made available as a binary. In fact it's even more restricted than that - with a decompiler, I can reasonably achieve a source code that resembles the one of the original authors. With an ML model, there is no way of reversing the "training" process.
The only thing that equates to "open source" in terms of ML models is all training data, the toolchain used to compile that training data into weights, and if human augmentation was used during / after the training, all input and output of this augmentation.
But no one of the large players will ever release that. First of all, the training data is heavily contaminated. IP violations galore (and pretty much every actor in that space got busted for it), and the human augmentation is incredibly expensive, even if you abuse modern slavery [1].
[1] https://www.theguardian.com/technology/article/2024/jul/06/m...
This was before llama4's lukewarm launch.
I do have a theory : Llama3.1 marks the point where Zuck got seriously interested and took over the reigns in driving the work. From the minute he started directing things instead of considering the AI work as a quirky side project, things went downhill. He tried to force a huge scale up in Llama4 which didn't work. Then as we know he disbanded the whole team and brought in a new crowd of mercenaries who may or may not have had the technical skills but they came into an organisation in disarray and still driven by Zuck himself who is continually forcing decisions that are not well founded in the science.
All the above is an entirely evidence free fan fiction version of things, but I would be completely unsurprised if it is true.
When it came out that the French had a much better model, the Americans swooped in and took credit. This is was the beginning of llama.
The Frenchies were predictably pissed and left Meta over the next few years, as RSUs vested.
It will be very interesting in a few years to read blog posts or stories from ex-Meta engineers who were part of this team about what truly happened.