Definition for Open Weights Licensing | Hacker News Reader

Definition for Open Weights Licensing

github.com

39 pointsandy9923 comments

This is one of the few places I would actually prefer the GPL. If someone contributes to this repository with it's current license a company can simply get the weights for free and sell them. You've just made it easier to create another OpenAI, not keep the weights open and auditable.

To me, the openness and auditability of weights is the most important. Right now we're heading quickly towards a world where CyberDyne dominates the AI landscape and you're either going to be with them or not involved. In that case, forcing a potential user to contribute weights back will create a more robust ecosystem for so-called personal AI. There is no better license for that then some version of GPL.

Yes I agree - I actually wrote something about this the other day. There's a need to think about how and when GPL style licensing can be applied to weights, and what terms make sense to accomplish the "software freedom" goals of the license while still keeping it usable. I'm not a lawyer, I've only considered it from the developer angle: http://marble.onl/posts/motivations_for_open_source.html

> a company can simply get the weights for free and sell them.

This is the flaw in your logic. If the weights are freely available, they can not be sold for much value. That will in turn mean there is no "another OpenAI", because the core product, AI, will basically be freely available.

Having the weights on their own doesn't mean anyone can have their own OpenAI any more than having a core Google index snapshot would mean anyone can spin their own Google.

The timescale difference between the need for weights taking account of recent events and Google searches is huge.

If that's the case then we might as well short the stock of all the public cloud companies who repackage and sell Open Source software. Why use RDS when PostreSQL is open source?

Well, I am arguing basically that basically AI APIs would be much more prone to commodification. There would still be companies, of course, but probably not with value on FAANG scale etc...

Certainty it would be easier to set up an API for a given model than providing what RDS does, to borrow from your example.

Because RDS comes with infra to support the software. This is what many want from their ai weights too. For it to come with infra to support the deployment of apps that use the weights.

> a company can simply get the weights for free and sell them

This is unironically best scenario for me. Multiple competing companies build infra that I can pay to use, instead of me building my infra that is powerful enough to run giant models.

A short while ago I wrote an AGPL-like license for model weights: https://freedevproject.org/faipl-1.0

I'm still suspecting that model weights aren't copyrightable, though.

Interesting question: To what degree does additional training obscure lineage of a set of weights?

As mentioned in the other thread, there’s not currently a consensus on whether copyright applies to the compiled weights in the first place. If not, there would be nothing to license. It’s quite possible the only way to protect the weights is as a trade secret: hidden behind an API, or with a confidentiality clause as part of a service agreement or the like.

Dunno who these guys are, but generally w.r.t. ML licensing, research kit is the important thing nobody is publishing. You know, the code used for diagnostics, visualization, benchmarks, decision support, etc. Research kit can be orders of magnitude larger than the final implementation code. Weights are just a binary with expensive build process.

Not just a binary because a common use is to fine tune from them to create derivative works. Look at all the models based on LLaMA.

Yes, precisely. If fine tuning is the preferred method of modification, then under the GPL the weights are source code.

Personally I think the truth is greyer than black and white.

Fine-tuning is like statically linking a large library (available in binary form only) to a small app. (An optimizing linker can blend app and library code so much they cannot be distinguished.) Weights intended for fine-tuning are still a binary, just a reusable one. Nothing new here.

This is a great idea, but I'm thinking weights aren't enough. If the recent rumors are true of the latest models like chat GPT-4, for example - they have a realitively new style of overlapping _a cohort_ of models, not just utilizing a giant matrix of weights (as has hereforto been the defacto architecture) - or is this still the case even for chat gpt 4 after training?

I'm no expert in ML, so perhaps I am ignorant as to what information is truly needed to reproduce results. I just cant help but think you'd need some hints at the model architecture itself as well.

There's nothing new about a Mixture of Experts architecture( and it's still essentially a giant matrix of weights). They just consistently well underperformed their dense counterparts so people were surprised.

But this recent paper shows MoE seem to benefit a lot better with Instruct training than their dense counterparts. https://arxiv.org/abs/2305.14705

If someone wants to release weights but attach a no commercial license to it I'm somewhat fine with that. I won't be able to sell it behind a shallow API but I'll be able to use the output of the model (that can't be prohibited) and sell that or do whatever I like with it. It's not as ideal as CC0 but I'm ok with the compromise.

> I'll be able to use the output of the model (that can't be prohibited)

If the model is copyrightable, they can prohibit you from running the model for a commercial purpose, even if they don't have copyright on the output.

Not sure why you are downvoted, this is exactly my concerns with llama. You can’t run llama with the intent of distilling your own commercial model.

(Maybe it's wishful thinking. But, precisely. My analogy would be: you can't run Visual C++ for commercial purposes without a valid licence, even though (ignoring linking against the runtime), the license of a compiler doesn't affect the licence of the output.

If weights are copyrightable, then you need permission to copy them into computer memory/run them, just like any application.

Discussion from the other day on a similarly themed article.

https://news.ycombinator.com/item?id=36601622

Reply on news.ycombinator.com