Hacker News Reader

Comment by charcircuit | Hacker News Reader

The weights are the preferred form for modifying or integrating with other models. There is no obligation in open source to transitively open source all of the documentation / tools used to create the open source project.

>a term that it seems nearly everyone has agreed is appropriate

Models being considered open source even if the original training code / data is not released also is something almost everyone has agreed to be appropriate.

View on HN

> The weights are the preferred form for modifying or integrating with other models

It's the 2nd time I hear this argument and I'm already fed up with it

Is it the preferred way only because training is expensive? It's like saying binaries are the preferred way of modifying program because you can't afford to have a fast enough machine to compile it yourself.

Most people don't have resources to compile their own browser, but what makes some browsers open is access to the source.

Maybe it's the preferred way because the people sharing models are themselves working with weights and no data?

> Is it the preferred way only because training is expensive?

No, it's the preferred way because that's literally how you train it. Contrary to popular misconceptions, you don't "compile" data into weights. You initialize a model (based on architecture, config, etc) and then you modify it via training. But crucially they (i.e. model creators) modify it the same way (technically speaking) as you would. That's what the license grants you. Nothing less, nothing more. The "how" as in knowhow has never been something covered by a license.

Open-weight is something dreamt up by people misunderstanding the basics of model creation and training, and having ideological biases against AI and/or LLMs. It is what it is, but you should know that you are technically wrong. A model released under an open source license (Apache, MIT, etc) is an open source model. Because the weights are the source of the models. Training is not "compilation". Training is the "how" as in knowhow to modify the model. Training deals with values. Source deals with everything, including values.

In the past, if someone would have released a piece of software (say a PID controller algo) w/ hardcoded values, no-one would bat an eye. LLMs are just that, with billions of hardcoded values. Nothing less nothing more.

> There is no obligation in open source to transitively open source all of the documentation / tools used to create the open source project.

Open source means open source code. Open weight means a binary file dump, not unlike an exe file. There is nothing open source about it.

Its like having a closed source text editor that censors certain words, and an open source text editor that censors certain words.

The latter can easily be recompiled, the former requires reverse engineering. Both may give you a license to use them freely.

Per the OSD definition of source code, "the source code must be the preferred form in which a programmer would modify the program." which means an argument could be made (as charcircuit is making) that the weights, being the preferred form to modify, are the source.

I do prefer open weights as being more precise (like, is it even really software that has source code in the first place?) but I feel like at this point the ship has sailed somewhat (though if this is something you're willing to spend your time arguing then... moral support I guess?)

You can also edit/patch binary executables, but no one says about that software that is open source.

You can, but its not the first preference

Incidentally, I do. I fucking hate the open source maximalists like Stallman who suggest that not having source code is equivalent to not having control over what runs on your computer. But you always[1] have control over what runs on your computer. Reverse engineer it if you care so much! Every form of DRM is inevitably defeated, because you do have control over program execution, regardless of access to the source code. Stallman's argument seems more lazy and entitled than principled.

[1] ...setting aside hardware attestation bullshit, which does need to die in a fire.

I think you are failing to see how the weights are the preferred form of working with a model. It's like if I shared a png with others to use as a meme template. Even if I don't share the source code to photoshop other people can use that "binary file dump" to make derivative images of their own with it.

I feel like the arguments being made about sharing the training data are from people who don’t have any experience training ML models and have never worked with weights. The biggest reason I have seen that people want the data is to complain about some aspect of it (copyright, bias, etc)

I would be very interested to hear someone explain what freedom it is they need the training data to enjoy.

Reply on news.ycombinator.com