Hacker News Reader

Comment by cranberryturkey | Hacker News Reader

How do you get ChatGPT to read the book?

For any book popular enough to have be put into GPT training data, you can get a summary without uploading it. The idea that these big, corporate models (their "intellectual property" if you will) have been trained on data they downloaded from ebook pirating sites... it's a real head shaker.

https://aicopyright.substack.com/p/has-your-book-been-used-t...

If these were scrappy start-ups looking to survive one can sort-of understand why they might bend the rules about copyright to train their models.

But we're talking about behemoth companies here, one of which deigns to make a pitch worth 7 Trillion dollars (10% of global GDP).

So... they're trying to rake in investment at levels that are unheard of in human history. Surpassing Apollo, the Manhattan project, the great pyramids of Egypt, the Great Wall of China, and anything else one might think of.

And they're not paying for the books ???

GPT-4 turbo has a context window of 128k tokens, so you can divide every book into a few chunks for initial summarization, then merge the results for the final summary!

Wow. Sounds like it’ll get expensive quick with all the api calls. I hope you make your money back via Amazon.

I assume you have pdfs. How do you split them up?

pdftotext is a Linux utility to extract text from PDF files. Then you can easily cut the text file into pieces. For best summarization, it's best that the pieces overlap a bit, so that important paragraphs that lie on the boundaries are not lost.

I'm assuming you don't need to. ChatGPT can just create the bullet points from summaries of the books consumed as part of its training data.

Reply on news.ycombinator.com