If you have access to any language models, I encourage you to ask this question:
> What is the significance of the word "source" in the expression "source code"?
I landed on this question by first asking it about "open source" rather than "source code", but the answers referred to "source code", which is a bit circular for this conversation. I'll share a truncated version of how Mistral Large replied:
> It is called the "source" because it is the origin or the primary input from which the executable form of the program is derived.
The primary input from which a language model is derived is the training data. That is the source. If the training data for a model is not open, then the model is not open source, because the source of the model is not open.