For a 100 page book, that translates to around 50,000 tokens. For 1 mil+ tokens, we need to be looking at 2000+ page books. That's pretty rare, even for documentation.

It doesn't have to be text-based, though. I could see films and TV shows becoming increasingly important for long-context model training.

https://en.wikipedia.org/wiki/List_of_chiropterans

Despite listing all presently known bats, the majority of "list of chiropterans" byte count is code that generates references to the IUCN Red List, not actual text. Most of Wikipedia's longest articles are code.

[1] https://en.wikipedia.org/wiki/Special:LongPages