A search engine for searching books in the Z-Library index on the IPFS network

Works great, that's old news which is no longer current.

ZFS (Zettabyte File System) is very much separate from Z-Library (and its alternative frontends like libgen).

I’ve dived deep into IPFS and built several prototypes on top of it. It ended up not being performant enough for me, and that was after heavily modifying the codebase so that it was true p2p browser & server (their webrtc transport had a lot of issues and they didn’t seem too interested in it, but my needs required it as a backbone).

The security was also a concern, and the scaling had issues. Pinning millions of small items got so slow it would not function. Then I ended up having concerns over hashed based addressing being easy to censor with the architecture IPFS was using (more hub & spoke than anything, given signaling and relay servers were centralized).

I could go on but I ran into so many issues I ended up implementing my own solution that did everything I wanted. Wanted to squeeze even more performance, I’ve been converting that solution to Rust.

This was a couple years ago so maybe things have changed since I used it. Last I checked, they seemed busy on Filecoin.

The idea of IPFS is great and I want to see it succeed, but I think that they got so caught up in their jargon and modularity, the project lost track of some fundamentals.

IPFS is not the only option available. Swarm's been making a lot of progress.

Hackernoon released a blog post comparing the two projects.

https://hackernoon.com/whats-the-difference-between-ipfs-and...

It seems case sensitive on some (or all?) of the input fields. At least for the author field in some of my searches.

Same here. Was working a few hours ago.

It's possible they were "hugged" enough to instigate a firewall rule dramatically cutting down allowed visitors, which would explain the Error 1020.

Might be worth reaching out to the site's creator -- info taken from their site [https://zu1k-com/about/]:

"If you have any questions to communicate with me, the preferred way is to comment directly in the comment area of the corresponding blog post, and I will receive an email notification

In addition, you can also reach me in the following ways:

Email: i@zu1k.com

Twitter: @zu1k_lv

GitHub: @zu1k

Keybase: @zu1k

Matrix: @zu1k:mozilla.org

My PGP public key can be obtained at https://pgp.zu1k.com or hkps: //keyserver.ubuntu.com

( 2A65 F6F3 1EDA D922 D7E7 E97B AE38 1A8F B1EF 2CC8)"

People should stop over-using CloudFlare for everything. IMO it is not good for IFS "hosting", either.

They act like gatekeepers. Along with blocking, as in this case, they make browser assumptions that force visitors to use one of the most recent mainstream "approved" and unmodified browsers.

For other browsers, they serve "Checking if the site connection is secure" with javascript fingerprinting and a captcha, and sometimes even solving the captcha won't let me in. i have also seen frontends misbehaving due to them not expecting cloudflare error response codes.

Some dvelopers apparenly just blindly enable cloudflare and don't care. :(

reported https://github.com/zlib-searcher/zlib-searcher/issues/57

I had the opposite... a book I was looking for on a lib gen front end a few days ago but could not find came up here (the Narada translation of Dhammapada if curious)

> libgen front-ends

Libgen is not the same as Z-library, there are many books on Z that are not available on libgen.

Z-library began as a Libgen mirror but split off and stopped sharing uploads. It's kind of scummy, but I guess that just means people need to put more effort into uploading its contents back to Libgen.

Large pdf and djvu files generally means low quality scans.

Pirates being picky, eh?

Don't get me wrong, I do appreciate the access that libgen and z-lib provide and think on the whole it is beneficial to society. It is still not paying for people's work though.

One mitigating factor is that it’s very easy to “correct” all but the very worst ebooks in Calibre:

- Metadata and a cover are one click away

- Removing excessive spacing, padding etc. is a few clicks

The books are free, if you download the wrong version of a book you can simply try another and it costs you no more than a minute of your time.

Do you know how it worked before we had the internet for books? You had to go to a library yourself, which may have taken an hour of your day just in the journey, then hope the library had the book you wanted. If not, you may have to wait weeks for another library in the system to mail the book to your library.

You might find this interesting:

Reverse engineering yet another ebook format (Nemanja Mijailovic) https://mijailovic.net/2022/12/25/hkpropel/

My layman's understanding of copyright law says you didn't commit a crime, but whoever sent you the book did commit a crime. My understanding is that it's not illegal to download a book, it's illegal to distribute a book. (Or other copyrighted work.)

That's really down to your personal ethics and how much you are willing to tolerate.

As you're not distributing it, the chance of someone coming after you for downloading a non-legal copy is almost non-existent, and pretty much 0 if you've taken some basic precautions.

It sounds like you've answered your own question there; that as you'd read the majority of the book, that there's no real loss to you finishing it for free.

Not an answer to your question, but interesting context for how eBooks work with libraries. And how not checking out a second time effectively is a loss of one “use” that may have directed the library to spend more on this book, rather than another. Sounds like you’re aware. Loved this Planet Money podcast

https://podcasts.apple.com/us/podcast/planet-money/id2907834...

I think the central ethical question to piracy is that ultimately, buying a book someone wrote is fair to the author. The author is part of a society -- our planet, did something to contribute to society, and it's fair that we reward that person if we also benefit from their work. Therefore, whenever you benefit from someone else's work, how do we reward them.

Now, I think this question is nuanced. For example, let's take Harry Potter. That book has sold millions. In my mind, J.K. Rowling has already been rewarded by humanity for her work many times over, so if you were to pirate her book then I think that is a far lesser evil than pirating a book written by an author who has barely made any sales.

Basically, I view the question of piracy as follows: (a) have we rewarded the author enough for that specific work? and (b) if not, then it's not right to pirate their book, and I should buy it. I think everyone should ask themselves that question.

The same with movies. Marvel movies are a money-making machine and they have more than enough money to keep them going. An indie film much less so.

The question of whether an author has been rewarded enough is tricky. For example, if Elon Musk wrote a book, I would have no problem pirating it in a microsecond because society rewarded him enough. But maybe that's not so clear-cut because the editor might be poor. Thus, I think it's safer to go on the basis of sales rather than the wealth of the author, except of course if the author is self-published. In that case, if they are rich, then who cares. Of course, what is rich? Well it's hard to give exact numbers but if they have 10 million dollars that's definitely rich enough.

> Is that a crime?

You didn't say where you are. Copyright infringement tends not to be a criminal offence, unless you're doing it as part of a business (so if you print out copies of that ebook and sell them) or you do so much of it you distort the market (you distribute so many copies of that ebook you destroy the profits that publisher might have made).

The rights holder can sue for their losses, but this tends to be the cost of the item, so they tend not to do that.

> Is that a crime?

In many countries it is at least a copyright violation, although it should be noted that downloading (not sharing while torrenting) is legal in some places.

> Have either the author or publisher lost money?

On average, probably yes, because some people would buy the book, some libraries would get more budget or allocate more of the budget to that book, and most importantly, if you did re-borrow it in 3 months it'd further expand the waitlist, perhaps making someone go "screw that, I'm not waiting 4 months" and buy it.

OTOH, if you took all the money being spent on entertainment (books movies etc.), collected that as a tax, distributed it to the entertainment industry based on what people actually consume, and in exchange made everything available to everyone, there would be no loss of revenue and everyone would benefit from much higher, friction-free levels of access.

That's part of what made Netflix work. It extracted ~$120/year of revenue and gave access to everything. Now, with all the competing services, they probably still don't extract more than one service's yearly fee over the same population (because few people have more than 1-2 services, and the ones that just gave up and went back to piracy probably make up for the ones that now pay for 2 instead of 1 service) but people now have much less access.

Without a massive "media tax" which would have practical issues that don't make it feasible or desirable, some friction on free access is necessary or nobody would pay for content, and at the same time that friction harms society by limiting access. Piracy puts an upper bound on how much friction can be added before people go the yarrr way.

Of the past twenty ebooks I've downloaded, 18 have dead authors and only one is alive.

Unless you focus on the very latest releases, most authors are already dead. If you round up just about any list of great works from the 20th century, you'll find that most of those authors are dead and have been for years. If you like 19th century books, all of those authors are dead.

Not sure why they have blocked access to me, i'm trying to connect from an Australian residential IP, AS4764

There is a discord server now, where you will find all your answers, plus some.

I've always been able to access it via tor which is operating normally.

Just be sure to confirm a disposable email address with them or they'll never let you in. Registering an account by itself is no longer enough when using tor and their website will block you and not tell you why.

Seeing that the US managed to extradite a British citizen from his own country for hosting links, I would definitely say so. The right way is to do this is to host this on tor or similar and only advertise it on the internet behind a good vpn.

It's ridiculous that this is what people need to do to help others, but that's what ridiculous laws and circus courts get you.

Anglo countries tend to aggressively extradite, even for things that are not major crimes by their own law, to the US over copyright related offences. I'd tread with extreme caution.

About as dangerous as using it because ipfs is NOT anonymous.

You may want to try building it locally without their AsciiFoldingFilter and see if it behaves as you wish: https://github.com/zlib-searcher/zlib-searcher/blob/0.6.0/cr...

I don't know tantivy well enough to know if it'd be possible to put the folded in one field and the literal in another and then use the search syntax to choose between them

Nothing wrong only the site owner block it temporarily surely with the large number of requests since sharing otherwise I shared in a previous comment another site to access the tool:

- https://news.ycombinator.com/item?id=34184167

Working on this now, cherry-picked audio from such AI systems abound, but submitting arbitrary text for audio synthesis requires a much higher quality bar to be listenable. Planning to have something out early 2023.

The Google Books mobile app will read any(?) book in your library, and at least they used to support uploading your own books, although I haven't tried it in a while: https://play.google.com/books/uploads?type=ebooks and in the individual book, tap on it, tap the bamboo menu, choose "Read aloud"

>You do not have access to zlib.zu1k.com. The site owner may have set restrictions that prevent you from accessing the site.

I think I hit the cf limit after 10 searches.

You mean, the owner has configured Cloudflare to aggressively restrict access

I believe it is indeed our index. :) Hurray for open source and open data!

https://zlibrary.to/ (to save a few guesses for interested minds)

If you like a book, buy it. I do. I love physical books. But, my interests are varied enough that I need multiple sources - sometimes I’m only mildly curious. Libby/Overdrive has been a lifesaver in this regard.

I would love to have a digital library of my physical books though, and that’s where this is coming in handy because I don’t want to buy it twice, especially when the ebook almost costs the same as the physical one. It would be nice if there was a “for ${1..5} more, get the digital one too!” - but then there’s DRM to fight with also.

I get the feeling that poor authors stand to lose the least from this kind of thing

There is no evidence that piracy hurts authors. It might hurt publishers (although finding evidence of that has been hard too).

So, you mean, the current copyright solution? That has seen creation of works skyrocket? Or do you mean a system where people can steal what they want because 'reasons' and 'they want to pay less because xyz'? Do you propose a system that takes ownership from authors and redistributes their works maybe? Why shouldn't owners get to set the pricing? Just because people want to steal doesn't mean it's ok to. Just because people don't like the price doesn't mean they should get to set it. The current system not working would look something like no works being released (because the author felt it wasn't worth it). The fact that people put in so much effort to steal these works shows the current system is in fact working. People wanting to steal does not equal not working. Authors can set their prices to free, but funny, they choose not to (as is their right).

> imagine this concept is being popularized at first in the modern era. I'm pretty sure there's a huge opposition

Video rentals have been popularized in the modern era.

Netflix mail rentals have been popularized in the even more modern era.

I'm sure some copyright holders argued that they should be burned down, but clearly there wasn't a "huge opposition" to it?

That is a crude rendition of the AI art debate, and also of piracy.

A public library buys the book. Many people can read it, but it guarantees one purchase, and often many libraries will buy the book. It also limits the amount of readers to those who live relatively nearby. Books were also traditionally written with the ideas of libraries in the author's mind, and also many authors benefited themselves from libraries.

AI Digital art has many facets which make it detestable for some: it floods the market with stuff that can be generated at near zero cost, it weakens the connection between people because people are no longer looking at something another human created, it cheapens the spirit of creation.

While libraries, Library Genesis, and piracy might have some facets in common, they are separate entities and should not be conflated in any sense.

Has LibGen already caught up to Z-Library? I thought that was still very much a work in progress.

I think the best thing about libgen is their detailed filter to search books with covers which makes it really easy to scan books you are looking for assuming you know the cover pages.

But there's no IPFS link there.

Of course there are examples on their [GitHub](https://github.com/zlib-searcher/zlib-searcher#original-sear...)

You can self-host it: https://github.com/zlib-searcher/zlib-searcher

No one is entitled to a particular form of revenue. We live in an attention-based economy. Talented and savvy authors can find any number of ways to derive revenue from the attention their work attracts apart from charging IP tolls.

If not, they can always get a job where they have no ownership stake in the fruits of their labor, like almost every other job and 99.9% of people writing code for money.

Strong agree, I still haven't forgiven myself for that time ABBA starved to death after I taped "Dancing Queen" off the radio.

I use OverDrive/LibGen etc because it's generally more convenient than the public library. If I like a book enough to actually finish it, I buy a paper copy because ebooks are inherently inferior and probably always will be. If I was still a student I would use OverDrive/LibGen etc specifically to steal from textbook publishers who charge $100 for an 18th edition riddled with factual and grammatical errors in hope that they would go bankrupt, but it wouldn't work. We have a long way yet to go.

Copyright enjoys stealing culture from everyone

I won't say if I have or haven't downloaded ebooks without paying.

But I will say that I have bought both ebooks and physical books from authors that I like and appreciate the work and effort they have put into their craft. I've spent rather a lot on it, actually.

A wealth-indexed fee averaging about $5/mo per household in the US and collected through broadband providers could provide access to every work ever published, pay authors, and eliminate both the need and practice of piracy.

The publishing industry prefers, however, to hoard its Smaug-gold, make criminals of the curious, deny access both to the vast majority of the population and to the vast majority of works, all to line its own pockets at an immense cost to the rest of us, authors, artists, musicians, and other creators included.

<https://news.ycombinator.com/item?id=33698018>

Nobel laureate economist Joseph Stiglitz: "Knowledge is a public good"

<https://archive.org/details/globalpublicgood0000unse/page/30...>

Copyright is brain damage:

<https://yewtu.be/watch?v=XO9FKQAxWZc>