Hacker News Reader

Comment by Chance-Device | Hacker News Reader

I agree with the other poster who said something to the effect that the model is open source and the released weights are, well, open weight. But the distinction is so trivial that I think it highlights the stupidity of this whole thing.

View on HN

The distinction in this case is decidedly not trivial. Open weights are great, we can fine tune them and mold them to our needs. But we don't have the training code, training data, etc, to be able to reproduce or tweak things at a more fundamental level.

"But not everyone is going to spend the money or time to train their own models from scratch!" I hear you say. And there's some truth to that. But if we had truly open source LLMs, then AuroraGPT development would be fast tracked and we would have a fully government funded scientific frontier model instead of only fine-tuning models.

I believe the training and inference code is open sourced as well, just not the training data itself, and I think we all agree, data != source.

Actually, in the case of models, training data is functionally ~= code, as the final item cannot be reproduced without it. And, as the open source definition states in its criteria 2: "Intermediate forms such as the output of a preprocessor or translator are not allowed."[0]. The intermediate output in this case being the weights.

[0] https://opensource.org/osd

I don’t think open source authors have a responsibility to also open source their binary. And if they do happen to provide a binary they aren’t also required to open source their compiler. Usually just the source code. I personally feel if they opened source only the code and not any model weights, it would still be fair to call it open source because “open source” refers to the code itself, not artifacts produced by the code, or possible inputs to the code.

It's a matter of intention. The intention of open source is to allow for anyone to be able to sufficiently reproduce some given artifact, at no/reasonable charge (as stated in the definition). Yes, the source code for a binary must be made available, if is to follow the OSD. No, the compiler doesn't need to be open sourced, unless it's an integral part of the program (a compiler would also have to be made available if there's none generally available for the source language).

This intention necessarily translates to models. In order to reproduce a program, the source code and a relevant compiler is what is required. In order to reproduce a model, the architecture (source code), training data and initial parameters are required.

Trivial? ."the model is open source and the released weights are, well, open weight." I'm not sure why you don't want open source model data or think open source is trivial. This sentence makes no distinction or point.

Reply on news.ycombinator.com