Open source means something else.
Calling released trained models open source is akin to releasing the binaries of a software for free, without releasing the source code, and calling it open source.
If it was trained on the entire internet for example - you can't just bundled up every piece of data on the internet into a zip file to include with the code to interpret it.
To use something like Meta’s open sourced LLAMA2 model you don’t need the data. The model is self contained. It’s a compressed lossy form of all the data it was trained on.
The weights allow you to continue its training with new data of your choosing.
This is a straw man argument as Meta has released the model weights and not just the trained model. When it comes to models the weights are the source.
There's a philosophy of open source based on recipients being able to make any small and large modifications to the source material as they see fit.
Just being able to build new projects with something or read what it does not qualify in that context -- it requires the freedom to rework the thing itself to suit your own needs and ends.
In the case of these models, that philosophy would have the source training data and the training algorithms themselves available, editable, verifiable, and repeatable. Ignoring the financial challenges, a truly open source model would let you remove, add, edit, or reclassify a sample in the original training data and then build new weights of their won.
The released weights are not that kind of open source, and that's what these arguments are generally making the a case for.
There are still plenty of grounds from which to critique that argument if you wanted to, but there's no straw man involved in it.
The reason I see it as a strawman is because the weights are the source of the AI model and they do allow you to modify the model however you please.
The data is not owned by Meta, they can’t release it and it’s not required to use or modify the model. This is like wanting a team to release the research that informed product development with their source.
The data and training code are not the same as the AI model itself. This is not the same as code that produces a binary, this is more like code that produces another form of code.
If you try to replicate the model you won’t be able to because the process is not deterministic. There’s no reason to replicate the model, you would fine tune it instead.