I want to believe most of what they're missing is material held in national and public university libraries and is material specific to the country the books or papers come from. Most of their collections are not digitized and that's what I keep seeing, and then some books that are profession-specific.

They're basically rebuilding Google books, and keeping it private. That's privatizing all of humanity's knowledge. Then from what I see some other missing books are just rephrasings of other books, which can already be done with synthetic data using another LLM to generate it. I would assume a lot of those books are school text books as well, which are almost never digitized and are rephrasings of university text books. Maybe they are not able to generate synthetic data of sufficient quality and are resorting to human books that rephrase information instead