Hacker News Reader

Comment by gizajob | Hacker News Reader

One of my favourite places on the internet too. The thing is, you just search for what you want and spend 10 seconds finding the right book and link. While I'd love to mirror whole archive locally, it would really be superfluous because I can only read a couple of quality books at a time anyway, so building my own small archive of annotated PDFs (philosophy is my drug of choice) is better than having the whole. I think it's actually remarkably free of bloat and cruft considering, but maybe I'm not trawling the same corners as you are. Do kind of wish they'd clear out the mobi and djvu versions and make it unified however.

View on HN

> While I'd love to mirror whole archive locally, it would really be superfluous because I can only read a couple of quality books at a time anyway, [...]

I'd love to agree but as a matter of fact LibGen and Sci-Hub are (forced to be) "pirates" and they are more vulnerable to takedowns than other websites. So while I feel no need to maintain a local copy of Wikipedia, since I'm relatively certain that it'll be alive in the next decade, I cannot say the same about those two with the same certainty (not that I think there are any imminent threats to either, just reasoning a priori).

Speaking of mirroring, is there a way to download one big "several-hundred-GB" blob with the full content of the sites for archival purposes?

Surely that would act as a failsafe to your problem.

I think it's split into a several different torrents since it's so big.

Well when a site claims it's for scientific research articles, and you search for "Game Of Thrones" and find this:

https://libgen.is/search.php?req=game+of+thrones&lg_topic=li...

Someone's going to prison eventually, like The Pirate Bay founders. It's only a matter of time.

First, SciHub != LibGen. Allied projects that clearly share a support base but not identical.

Second, please provide a citation for the assertion that sharing copies of printed fiction erodes sales volume. At this point, one may assume that anything that helps to sell computer games and offline swag is cash-in-bank for content producers. Whether original authors get the same royalties is an interesting question.

Third, the former Soviet milieu probably isn't currently in the mood to cooperate with western law enforcement.

Even what you call LibGen isn't LG. These are LG forks, actually running against LG and pretending to be LG. LG was set up to create other libraries on its basis. Each of the forks aggressively fights for own dominance in all ways, and they resist the development of other forks by naming themselves LG and sucking in all the funds to personal possession without public reporting. Being forks themselves, they have closed the open project for own ambitions and for money grab.

Their values are incompatible with LG, and all what's left similar is the external part of letting download books, without which there would be nothing useful to look at.

Yeah, and the herculean work is actually done outside such aggregators by myriads of smaller collections, digitizing, binding, processing, collecting, and channeling millions of handmade books into rivers of literature, for free and ready to grab. The growth is global and isn't relevant to what the forks do.

Sorry to tell.

One might suppose it's not organizationally transparent for good reason.

It used to be for a good reason, indeed, but not any longer.

> djvu versions

This would be disastrous for preservation. Often the djvu versions have no digital version, the books not in print and the publisher isn't around. The djvu archives are often specifically because some old book, really has and had value to people.

Yeah, I always convert DVJU to PDF (pretty easy) but it never compresses quite as nicely.

DJVU is pretty clever in how it uses a mask layer for more efficient compression, and as far as I know, converting to PDF is always done "dumb" -- flattening the DJVU file into a single image and then encoding that image traditionally in PDF.

I wonder if it's possible to create a "lossless" DJVU to PDF converter, or something close to it, if the PDF primitives allow it? I'm not sure if they do, if the "mask layer" can be efficiently reproduced in PDF.

It can be done with relative ease. There is a commercial tool somewhere that does it, because I've run across many PDF files that use a DjVu like structure for scanned books.

It won't be perfectly lossless, because the IW44 compression of the color layer will need to be recompressed as JPEG or JPEG2000. The JB2 mask layer can be losslessly recompressed as JBIG2 or CCITT G4 Fax.

If you smoke enough algebra, you could use the DJVU algorithm to implement DJVU in PDF with layers. Or heck you could do it in SVG.

You can't do this in either PDF or SVG, except using JS, which many PDF and SVG viewers don't support.

I think more often than not djvu blurs out half-tone which can be too aggressive for e.g. newspaper scans and makes a blurry mess.

Why would they clear out djvu? It's one of the best/most efficient storage format for scanned books.

I'm not for clearing out djvu, but it sure is frustrating when a PDF isn't available.

It's not just about laziness preventing one from installing the more obscure ebook readers which support djvu. It's about security: I only trust PDFs when I create them myself with TeX or similar, otherwise I need to use the Chromium PDF reader to be (relatively) safe. I don't trust the readers that support Djvu to be robust enough against maliciously malformed djvu files, as I'm guessing the readers are implemented in some ancient dialect of C or C++ and I doubt they're getting much if any scrutiny in the way of security.

It's super easy to convert a DJVU file to PDF though. There's an increase in filesize but it's not the end of the world.

And since you're creating the PDF yourself seems like you can trust it? Since nothing malicious could survive the DJVU to PDF conversion since it's just "dumb" bitmap-based.

DjVu also contains text.

If your DjVu file contains an exploit for your DjVu decoder, even if you run it in a bombproof container, it could still conceivably inject malicious code into the resulting PDF file. That sounds far-fetched because the exploit payload would need to recognize that a PDF conversion was going on and respond by generating the PDF, but I remember when people thought exploiting buffer overflows was implausible, and this is not the same level of rocket science.

djvu is really quite a marvellous format, but I'm only able to read them on Evince (the default pdf reader that comes with Debian, Fedora, and probably a bunch of other distros). For my macbook I need to download a Djvu reader, and for my ipad, I didn't even bother trying because the experience would likely be much worse than Preview / Ibooks.

DJVU is supported by numerous book-reading applications, including (in my experience) FB Reader (FS/OSS), Pocketbook, and Onyx's Neoreader.

As a format for preserving full native scan views (large, but often strongly preferable for visually-significant works or preserving original typesetting / typography), DJVU is highly useful.

I do wish that it were more widely supported by both toolchains and readers. That will come in time, I suspect.

Calibre supports djvu on any platform. Deleting djvu books just because Microsoft and Apple don't see fit to support it by default would be a travesty.

Apparently you can install Evince on MacOS as well. But I haven't tried it there.

Evince doesn't come by default with Archlinux (my desktop distribution of choice), but I still install it everywhere.

> Evince doesn't come by default with Archlinux (my desktop distribution of choice)

This doesn't make sense; nothing comes "by default" on Arch, but evince is in the official repos as far as I see.

A few things come by default on Arch. See the list at https://archlinux.org/packages/core/any/base/ (many some of these entries like coreutils expand to more packages).

Yes, evince is in the official repos. Just like Chromium and Firefox. Or bash, but not any other shell (as far as I can tell).

Is there a torrent available that would allow straightforward setup of locally storable and accessible Libgen library? For the storage rich but internet connection reliability poor, something like this would be a godsend.

They have a dedicated page where they offer torrents, so pick one of the currently available hostnames: https://duckduckgo.com/?q=libgen+torrent&ia=web

Obviously, folks can disagree on the "straightforward" part of your comment given the overwhelming number of files we're discussing

My comment about djvu was mostly just about my own laziness, because (kill me if you need to) I like using Preview on the Mac for reading and annotating, and it doesn't read them, and once they have to live in a djvu viewer, I tend not to read them or mark them up. Same goes for Adobe Acrobat Reader when I'm on Windows on my University's networked PCs.

I wish they'd clear out the PDF versions and replace them with DjVu versions. DjView is better than any PDF reader I've used, and DjVu files are smaller than scanned PDFs.

That’s not going to work, because outside of e-book enthusiasts, few know what DJVU is and even fewer have the technological skills and will to figure out how to open it. A large part of LibGen’s demographic are university students downloading exorbitantly priced textbooks, and given that, having both a pdf and a djvu available would be ideal.

GNOME-based Linux distributions ship with DjVu support by default, and so do MATE and KDE and most document viewers for Android. But even if you're not using Linux, if you're going to spend 50 hours studying a textbook and you're part of a learning community like a university class, with dozens of people facing the same problem, one of you can spend 0.5 hours figuring out how to install DjView so you can read the textbook. That's a much easier problem to solve than finding out about Library Genesis in the first place, not to mention fixing your legal system so it's legal.

Reply on news.ycombinator.com