Hacker News Reader

Comment by fragmede | Hacker News Reader

Naturally, the full text of Harry Potter would appear in the training corpus, but why would frequency matter, and why would multiple copies get put in there intentionally?

View on HN

Naturally? It seems like that last thing I'd expect to see in a training corpus is a copyrighted work which is impossible to procure in electronic format, plain text. Did it scan pirate sites for those too? Surely OpenAI does not purchase vast amounts of copyrighted corpora as well?

Surely the most logical things to train on would be all the fandom.com Wikis. They're not verbatim, but they're comprehensive and fairly accurate synopses of the main plots and tons of trivia to boot.

Even if the full text is fully deduplicated, there is just so much more content about Harry Potter on the internet. And not just retellings of it, but discussion of it, mentions of it, bits of information that convey context about the Harry Potter story, each instance of which will help further strengthen and detail the concept of Harry Potter during training.

To add on to this, OpenAI definitely tips the scale in terms of making sure it doesn't make mistakes proportional to how likely people are to ever run into those mistakes. If it failed at Harry Potter, there's a lot of people who would find out fast that their product has limitations. If it fails at some obscure topic only a niche fraction of nerds know about, only a niche fraction of nerds become aware that the product has limitations.

Reply on news.ycombinator.com