Hacker News Reader

Comment by avereveard | Hacker News Reader

Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.

View on HN

I wonder if when you train a model based on the weights of another model, do the owners of the original model have any sort of copyright claim? They trained it on data on which they had no copyright claim either, fwiw.

Why bother with the weights when you can just take the literal output of another model and consume as part of your training corpus?

What kind of copyright claims can be made against the content generated by a model wielding the desired weights?

According to the US Copyright office's recent ruling, nothing generated by these models can be copyrighted (!), so in theory you should be fine!

nit: that wasn't a "ruling" so much as it was "guidance" [0] [1]

[0] https://www.federalregister.gov/documents/2023/03/16/2023-05...

[1] Discussed on HN: https://news.ycombinator.com/item?id=35191206

The position of the US copyright office is based in large part on prior case law (from your first link).

They're the first word on this, not the last, so it doesn't mean much. It also clearly leads to absurd results for text models, which can memorize their training data, or echo back your inputs. Or you could pass their output through a human to copyright it.

Not all countries have the same view on this either. A few countries, including the UK, maintain the view that the AI's operator, owner or custodian hold the copyright instead.

Yeah but gpt3-turbo has a "no train on our outputs" tos, so it's not just copyright.

Does it actually? The only thing I ever saw was "don't use our models to build competing models". Which is clearly ambiguous at best when dealing with generative natural text. This also implies to me that using their models to help with fine tunes of their other offerings would be ok.

Let's say company A uses OpenAI models to create an open dataset, like alpaca. If company B uses that dataset to fine tune a model, is company B liable to TOS violation? Afterall, they may not even be an OpenAI customer.

In fine-tuned models, for example, you may only train a real subset of the weights. Who is the copyright owner now and what is the justification?

There might be other ways of transfer-learning where you can greatly benefit from the pretrained weights, some of which may have not yet been discovered.

> In fine-tuned models, for example, you may only train a real subset of the weights.

I am not sure to what degree this applies throughout the various offerings. We are finding that "fine" tuning cuts pretty damn deep. For instance, with OpenAI you can start with base Curie/Davinci and build an effective chat bot that makes ChatGPT look incompetent in a very narrow vertical slice of some specialized domain.

As I understand it, it's not a copyright issue, it's a terms-of-service issue. You're only authorized to use GPT in accordance with their terms, which prohibit using the service to develop a competitor. Doing so anyway is unauthorized use of the service, which is a Computer Fraud and Abuse Act violation, independent of what is and isn't copyrightable.

This is an Americanism. In many parts is Europe copyright claims over copyright from the US are honoured, but yet, violation of TOS by itself does not put you in risk of criminal lawsuit.

There are also plenty of places around the world where US copyright is not honored.

This includes, arguably, the US itself, though copyright violation is a lot more blatant in some parts of the world, which don't even offer the fig leaf of trying to enforce US copyright in their legal systems.

Technically only three:

https://legalbeagle.com/6780679-countries-copyright-laws.htm...

One, the Republic of San Marino was founded in 301 AD and thus has its own laws that predate the US copyright laws by some 1400+ years.

Is there any reason their near 2,000 year old freedoms should be constrained by the whims of a teenage republic?

Just make your millions and fight in court for a hundred years whilst they judges spend years learning ML.

Are google and oracle are still fighting over java?

There's probably not a lot of hope for the litigation on hyper niche bits of config.

Hopefully someone will be brave enough to fight Meta on the LLaMA copyright issue. My concern is that the people who can afford a massive legal battle can also afford to train their own model.

Given recent guidance from the US Copyright Office, I doubt there are real copyright issues or enforceable restrictions on commercial use.

I would say that someone trying to sell the original model would be the only case that could have a real challenge, and even then its not that clear cut to me.

Everyone else using these fine tuned models, I don't think there is a case. Everyone's making non-copyrightable scaffolding. Read the recent USCO guidance.

You can monetize and sell things that have no copyright protection, you just can't prevent others from doing it too. aside from trying to make it inconvenient with the packaging.

There’s a big difference between copyright for material produced by a model, and copyright on the software and weights for the model.

As far as I know, USPTO has said that works crested primarily by AI may not receive copyright protection, but I don’t think there’s any indication that the software to run AI is not copyrightable. Maybe?

The code to train and run the model perhaps, but model weights lack two fundamental requirements available in normal code: human authorship and creativity. Because these models were the result of a purely mechanical process and thus lack these properties, they can't be eligible for copyright protection. At least not under current US law.

That said, regulatory capture is strong in the US. It would not surprise me if we see these laws changed.

We clearly need to run the weights _through_ a GPT model because that seems to strip copyright from them :D

You can use the summarize, generate technique to strip copyright from source code too.

You want people brave enough to fight meta for control of the work meta did and shared for free? It's attitudes like this that mean LLaMA will serve as a lesson for other companies to not share as much. It's too bad, because in normal open source, I like that the community usually tells people to abide by the license terms chosen by the authors. But in this, everyone's just demanding unrestricted use.

There is a real concern that this particular technology, as powerful as it is, will be kept behind the doors of the biggest corps and information gleamed from prompts will be further used to our collective detriment.

I am personally still coming to terms with it, but it is not just wanting stuff. It is making sure society does not get too separated in terms of power ( because it is already pretty stratified ).

I agree. I think that's why it's important that the lesson isn't "if you give an inch, the community will take a mile." Otherwise, they will keep it behind closed doors.

While they'll release useless minor models here and there, I think these companies will keep this tech behind closed doors regardless of what the community does.

Humanity needs to act quick in order to avoid yet another extreme consolidation of power.

Even if they wanted to keep it behind close doors, it could easily get leaked.

I've assumed llama was intentionally public to "commoditize the compliment"

The question is, who's going to stop us from using it for commercial grounds. Most companies that would use these models would be startups and are years away from any sort of potential audit that may occur.

Won't stop those that are willing to pull an Uber or Nathan Fielder it. It's not commercial, it's a parody or a non profit organization.

Da Vinci had no licenses. Nor Euclid, Nor Einstein.

In the long run, licenses are for losers.

And I think we can all agree technical progress under the copyright system is moving much slower than it did back in da vinci's day....

If only we didn't have patents holding us back we would be investing way more in R&D...

Fine, a modern example: Tim Berners Lee didn't need licenses.

Name me someone alive who's innovations are more impactful than his.

In the long run, licenses are for losers.

I can’t tell if this is satire, but i literally have autism

OpenAI keeps text-davinci-003 proprietary.

He meant the actual guy

Yeah, and there's a reason none of those guys were as rich as e.g. Mark Zuckerburg.

Ok Dude.

Reply on news.ycombinator.com