Authors say OpenAI 'ingested' their books to train ChatGPT | Hacker News Reader

Authors say OpenAI 'ingested' their books to train ChatGPT

businessinsider.com

46 pointsbratao47 comments

It's really clear to me that a new legal framework will be required to deal with the societal consequences of advanced AI. For example, if this article were instead about, say, an actual human who read all of these books, and then posted reviews and some summaries online, and then this human got sued by the authors, I think every single comment here would be decrying this as an abuse of copyright law, and that this should fall squarely under fair use.

The thing that is "scarier" if you will about what AI can do is the sheer speed and breadth that it is capable of. It really is the scale that changes how people feel about these technologies, and that requires new legal frameworks in my opinion because I think people really feel what is "fair use" is different if it comes from a person vs. a machine.

Similar analogy: before the Internet, pretty much everyone agreed you didn't have an expectation of privacy if you were walking around outdoors. But there is a marked difference in thinking "Yeah, I expect other people walking around may see me, or even take a picture of me" compared to "I think someone should be able to take a picture of me and put it on the Internet with my accurate geolocation so the entire planet can look it up, for all time."

Essentially they want a copyright on knowledge, Such a law can be misused. Also its hard to prove this just by looking at the weights, and for something as large as GPT-3.5 it would not be easy to prove this.

You’re right. Knowledge provided by authors isn’t unique and as long as AI doesn’t literally cite complete chapters word for word, this doesn’t seem to have anything to do with copyright.

To be fair though its an Open secret they trained it on Common crawl G4 and Pile databases. But proving it is another thing. By the time someone develops tooling to prove such things we will have an explainable AI.

>They argue that ChatGPT's ability to produce detailed summaries of their works indicates their books were included in datasets used to train the technology.

This just isn't true. The authors have detailed summaries of their books on Wikipedia, that claim seems unsustainable.

There are actually interesting legal questions about AI, but this case seems not that interesting. I don't even see how the authors would demonstrate that their copyrighted works were used and not some summary, sourced from elsewhere.

It seems relatively straight forward (famous last words) to assess whether actual copyrighted text is embedded within the network. If you can prompt output that includes verbatim extracts when the copyright avoidance post-processing is disabled then you know that it has been consumed.

Of course whether that was purposeful or inadvertently as a part of the larger training set would not be determined but you would know that the text is in there.

>If you can prompt output that includes verbatim extracts

If I create a program that picks random words from a dictionary and I end up with a seed that generates that text verbatim, then does that mean my program contains the copyrighted text?

You might be able to craft an intricate prompt that just happens to recreate that copyrighted text. Run it enough times until you get it verbatim and done.

>If I create a program that picks random words from a dictionary

And LLMs do that, except prior to picking the word, they do complex statistics to figure out the probability distributions of those words.

Almost certainly some combination of input and RNG seed will produce any "small" combination of words.

> you can prompt output that includes verbatim extracts when the copyright avoidance post-processing is disabled then you know that it has been consumed.

No, you know that likely that part was consumed. You would need to show that it will generate arbitrary passages from the text.

And LLMs are inherently random, so proof that this happens is very difficult to obtain and showing that it is actual output nearly impossible, especially if you just have API access and can't use the model directoy (e.g. fix the RNG seed).

If you have that you can debate if it is/isn't fair use.

Arbitrary passages is what I meant by "verbatim extracts."

Ok. I may as well come clean. For the past 36 years, I have been ingesting all of the information available to train a model called b33-j0r

I didn’t mean to read your books, they were mostly trash. I’m so sorry for reading things you published!

I hope this point of view wins eventually. It's the only sensible one.

Presumably you paid for some of the trash you read, either through buying a book outright, borrowing from your library (taxes), etc?

Hey, if we have to pay a content-agnostic version of spotify $0.000006 cents per token to a vague group of original authors, I’m all for it!

But how are we going to compensate Kurt Cobain, the Beatles, Beethoven, and random sounds I heard from my backyard for my supposedly original works on my soundcloud?

Snark aside, for this analogy to work you'd have to be able to leverage your "training data" at a much larger scale than a single person typically can. You would also need to be able to train at the same speed. Scale is the key factor here.

> They argue that ChatGPT's ability to produce detailed summaries of their works indicates their books were included in datasets used to train the technology.

If ChatGPT can reliably answer "what happens on page (x) of book (y)?" that would provide fairly convincing evidence, as summaries or study notes on a random book are unlikely to be that detailed. Moreover, if this approach worked consistently, it would enable the entire book to be summarized - or even rewritten - page by page.

not sure i agree with your conclusion since LLMs can easily avoid mentioning the page even if they knew it. Though id also guess that the llm didnt ingest page numbers and just ingested books raw? Anyway i tried it.

> im looking king for a quote in the book iRobot, what page is isaac asimovs the laws of robotics on ? provide the version of the book so as well so were clear

ChatGPT I'm sorry, but as an AI text-based model, I don't have direct access to specific book editions or their page numbers. However, I can provide you with general information about Isaac Asimov's "I, Robot" and the Laws of Robotics.

it doesnt know, but i also didnt coax it or jailbreak it to circumvent any avoidance

> it doesnt know

Correct, because it only knows the text, not the layout. It will tell you that it appears in Runaround, and that the Laws are "usually presented towards the beginning of the story, in the dialogue between the characters Powell and Donovan".

Ah interesting. That could be avoidance though. Guess it's useless for pseudo-copy protection schemes for some old games like "what's the first word in the third paragraph on page 17 of the game manual."

fwiw ive asked it to summarize the first 5 chapters before, and it was able to do that. Just a high level breakdown

There's a lot of discussion about whether existing copyright laws apply to training data, but not enough discussion of what should change in copyright law to give the best outcome to society.

I don't think treating ML systems as legally equivalent to a human brain is right, but I also don't think that copyright law is sufficient as-is. This is something entirely new.

It seems like society would benefit from have ML systems around that can be trained on copyrighted material, but not if it prevents authors from being able to make a career out of producing great work. We need to balance the outcome for owners of ML companies, authors, and society as a whole, ideally prioritising the latter.

What do you think we should do?

Considering that I'm not allowed to copy any of the physical books, films or songs that I own as it breaks copyright then copying it and putting it into a generative AI should also be illegal. Just because you change it from words into weights, it's just another form of compression. I'd rather not live in the world where AI models have better rights than people.

>Considering that I'm not allowed to copy any of the physical books, films or songs that I own

You should be able to and you can in some/many countries.

Can you point to any substantial reproductions of copyrighted novels that are output by LLMs? If it’s just compression, you should be able to pull a substantial reproduction of the works out of them.

The same thing we did when basket weavers were upset robotic manufacturing took over. Let supply and demand decide what society considers valuable. If we want the $2 basket that gets delivered yesterday but breaks in 1 year, so be it. If the master basketweaver thinks there is a market for selling human made baskets at $100 a pop, but they last 10 years, then prove it.

>but not if it prevents authors from being able to make a career out of producing great work

Why? Why should we hold back technological and social progress because it might make it more difficult for some authors to have a profitable career?

Are only people that have the mechanical skill to draw and write well the only ones that should be allowed to profit from creative work? Because that's how it's been so far. You might have the best ideas for a story, picture or movie, but if you lack the mechanical skill to bring them to life then your creativity is worthless. AI generation can help those people have a chance too.

Fundamentally this is about wealth distribution. The society do want those data to be used for AI. It's just the data providers should be compensated, instead of OpenAI just gatekeeping their product that critically depend on those data. The original model of copyright just doesn't work anymore in this new era.

Music streaming has changed the music industry. Can we expect something similar happening in AI?

Well, to become an expert in any topic, humans have to go to university and read textbooks as a part of the course. Do these humans owe anything to the authors, other than the purchase price of the books? Why should it be any different for AI LLMs/reasoning engines?

An LLM isn’t a human brain and it’s almost always a mistake to think of it as one.

The LLM is more akin to a search engine with a much more advanced interface. If Google published the entirety of a book without having the rights they’d be taken to court too.

Don’t get me wrong I’m not saying they’re the same, I think there are convincing arguments to be made either way. I just don’t think the comparison to a human brain is the right one.

which, for anybody following along at home, is related to something that happened back in 2015.

Google was trying to digitize the world's books and got sued for it.

https://www.npr.org/sections/thetwo-way/2015/10/16/449172748...

a LLM is closer to a very primitive brain than a search engine. A search engine can't make mistakes, it output data verbatim, it may not be able to find something, but it's never making mistakes. A LLM have an input, and an output, which can wildly differ, it can respond to your demands, even trigger other software with relevant data.

It's way closer to a primitive brain than to a search engine.

Pretty sure you answered that in your response.

For one thing, a single humans knowledge cannot be scaled to serve hundreds of millions of users simultaneously.

I still don't see why the person or company who owns the LLM would owe the book author anything more than the price of the book. Its not like the LLM is going to output or reproduce the book in its entirety. You'll ask a relevant question, it will answer based on the data it has, not just whats in that book.

Same as if you ask a human a relevant question about a book they read. Like an LLM, they can't print you a full copy of that book. They can only answer the question based on their knowledge of the book and other things.

What are we going to class these AI's as though?

If I read a book and then tell someone else about it, that isn't copyright infringement, if the AI 'reads' the book and tells someone else about it, they are claiming it is?

Here's an idea for a dystopian sci fi novel: AI learning is treated differently from human learning, therefore we start building computers with human/animal brains.

"If I read a book and then tell someone else about it, that isn't copyright infringement"

If I throw my computer in the trash can it isn't murder, but throwing a baby in the trash can is! Totally wacky, right?

Japan has allowed us to do this legally. Going to be interesting over here.

Same old Murican business model. Bill Gates did it, Zuck did it, Jeff did it, Elon does it. Embrace Extend Extinguish, Facebook forced tracking into the web and made profiles for people who didn't sign up, Google absorbed everything, ect.

The American business model has been Robber Barons. Since the 1800s, at least. OpenAI is the latest. The reality is, you can never give anything valuable to the 'free market' and an internet connected device, because you're never going to be able to govern it according to your values.

The problem is that this isn't only limited to OpenAI though. The same rules will also bind efforts like StabilityAI that does give away their models for free for everyone to use.

And digested them as well. We are currently in the post-digestion stage.

I could produce similar summaries based on the summaries of these books and discussions of them that are posted online. I think the easiest defense is to show that all the information shown in the plaintiffs' exhibits is also freely available online.

Reply on news.ycombinator.com