Hacker News Reader

Comment by famouswaffles | Hacker News Reader

https://crfm.stanford.edu/2023/03/13/alpaca.html They say they'll release the weights. But Of course, you'll be beholden to a non-commercial license. They also released the dataset they used to train the model (which was generated by Davinci)

I wonder why Meta didn't just train the 7b and 13b models on the full 1.4T tokens. They curves show they weren't even close to converging. Shame really.

View on HN

I am not a lawyer, but the consensus (around here anyway) seems to be that weights cannot be copyrighted since they are the output of an automated process, so perhaps the license cannot limit how the weights are used. The code can be copyrighted, but the code can also be reimplemented from scratch and used to load the weights.

Just because they cannot be copyrighted doesn’t mean that they cannot be considered a trade secret. The agreement for using the product could done with an NDA. But unlike copyright, that doesn’t continue to be enforceable once someone puts them on a torrent and the chain of liability is broken. /notalawyereither

Yeah, this happened with RC4 cipher which used to be a trade secret until someone posted it on Usenet: https://en.wikipedia.org/wiki/RC4#History

Well, trade secret law requires the owner of the trade secret to take steps to keep it... secret. Facebook has done the opposite, and they haven’t made the people they distributed the weights to sign an NDA.

And Facebook employees openly discuss the torrent magnet links.

God damn I hope this is where IP law lands. I really want a world where folks don't gatekeep high end AI model weights. Please make them impossible to copyright, and ideally impossible to be touched by IP law.

Why? I mean, do you feel differently about every other patented invention or IP? That would be weird.

That seems like wishful thinking. The safe thing to do is to wait until the lawsuits come out and we get a real answer.

Or if you’re not risk averse, then go for it and take one for the team!

The US doesn't have database copyright, but some countries, including the entire European Union, do.[1]

[1] https://en.wikipedia.org/wiki/Database_right

Interesting. That could be a huge disadvantage for the EU and it's companies that want to use AI and also often can't give their Data to third parties. But I am not sure a neural net is even considered a DB. Because when this was created they clearly had structured Information accessed by people in mind. There has to be some distinction otherwise I make a db that stores digits of PI in a file and if you publish part of it, that is considered substantial enough, you are infringing on my DB rights.

If I copy your database, that violates copyright. I generate the same data using my own process, it doesn't.

That's different from creative works where it's implausible to claim that you created the same work independently of someone else.

Those protections rely on "systematic arrangement", i.e. that some effort has been put into finding, verifying and collection a set of facts / individual pieces of data. It is not obvious that would apply to the set of weights that come out of a training algorithm. (standard I'm not a lawyer, I don't know but I suspect it is unclear enough that it would need to be argued in court).

I assure you that if you scramble the weights of a neutral networks, the model will perform poorly... Training something new cost millions of dollars in accelerator and research salary, and evaluation ('verifying') is a big part of the work.

This is true, but not currently relevant. The cost (difficulty) of training does not indicate how good a fit the model is for copyright protection.

I suspect that part of the arguing this out in court will be making those economic arguments as part of an attempt to extend copyright protection to model weights - but that is a separate issue from if they are protected currently.

Part of the economic argument for copyright is to encourage the investment of time into producing new works, so it may be a persuasive argument for extending protection. Although (relatively) recent history suggests that persuasive argument is not as powerful as giant wads of cold hard cash in extending copyright coverage.

Not copy right, but doesn't mean this isn't someone's property.

Information can’t be property.

IP “rights” are a perverse legal fiction that have inflicted untold damage.

I am curious how you are expected to prove in a court of law that the weights you have are derived from some licensed weights if I take just a few gradient steps from the original weights.

Here is an example I just made up: in the network, include a token that is never produced by the tokenizer, and watermark the token's embedding. It won't be touched by normal fine tuning and will remain the same.

I am sure there are better methods.

Same way that court determines whether my book is a paraphrase of yours. Substantial similarity.

I'm curious how they will be doing that for weights though. Euclidean distance?

They probably did but didn't release them for the public.

Why would they withold those ones but the the 65B ones? Seems like a weird choice IMO.

Reply on news.ycombinator.com