Hacker News Reader

The title got me really excited that they were doing full text search. Boy that would be an awesome project. Zlib and Google Books do it, but it would be great to have a open source version that everyone could contribute to, and provided access to full texts

View on HN

OpenLibrary does provide search access to full texts. For example: https://openlibrary.org/search/inside?q=%22institutional+thi...

It is open source and they're always looking for contributors. I think they'd especially welcome help improving search!

https://github.com/internetarchive/openlibrary/

I think a distributed OCR project is needed. Problem is that a lot of books are PDF scans and missing raw text. OcrMyPdf does a pretty good job of is but it's cpu intensive.

I'd wager that there are several players in the AI market who have already scraped and OCR'd every book and magazine on zlib and libgen to feed into training models. Google are almost certainly piped everything they have in Google Books into their models, before some future legal case says they can't. Won't take long before the open community starts doing the same.

Reply on news.ycombinator.com