Google Books (or similar) all book scans – $200k bounty (2025)
software.annas-archive.gl
software.annas-archive.gl
If it were not for Anna's Archive and Z-Library, I would've never been able to read the books that shaped who I am today, or keep my passion for learning alive.
Thanks, AA and ZLib! (Also, thank you to the authors whose books and knowledge I consumed without being able to pay them back.)
This is key for getting epubs to your Kobo.
I've been using MoonReader for many years now and settled on pretty good parameters that make the reading experience very comfortable on both my phone and my tablet.
I download epubs from zlib but then they're on my phone and transferring them to my Kobo is arduous. This makes it easy.
If you mean stripping drm I used Calibre for that but mostly I just avoid buying books with drm where possible.
I can't find the post but years ago on Reddit an author posted stats showing when her book turned up pirates online, real sales for it collapsed.
Because of this I make a point of buying books, programming books especially. Yes I download pdfs, I use them as previews. This has led to buying way more than I would have.
Anyway, I appreciate this doesn't apply if you live somewhere that these books can't be purchased. But everyone praising these sorts of sites tends to look at them from only a positive perspective.
I think that's at least a bit debatable. People thought that about (normal) libraries back in the day, but it ended up having the opposite effect.
Not to mention out of print books or academic books which is a big usage of sites like these, since lots of people prefer physical books and only reach for pdfs as a last resort.
(Archiving culture alone is not the same as also enabling universal access to the culture and knowledge one is acting as custodian for and serving to global citizens)
The Internet Archive has lost its appeal in Hachette vs. Internet Archive - https://news.ycombinator.com/item?id=41447758 - September 2024 (793 comments)
https://archive.org/details/brewsterkahlelongnowfoundation
Totally unrelated: Dweb camp 2026 is coming up for those interested: https://dwebcamp.org/
(no affiliation with any person or entity mentioned in this comment)
I remember opening "thinking fast and slow" and noticing the weird paging. After checking the official version's page count and seeing how the version in my hand doesn't match, my best guess was that someone had printed a pirated epub version.
I'm not part of the market for these products. I don't have access to them nor even if I buy some imported (probably illegally and by single persons) or printed version, am I going to benefit them since I'm disconnected monetarily from yhe author.
Me reading pirated versions of these books has no negative effects on the earnings of the authors.
I’m kidding. Knowledge should be free. It was never created in a vacuum. It belongs to us all
This is a common perspective on HN, but it's so jarring. Someone violates an open-source license and we grab our pitchforks. Someone pirates books and it's fine - really, the authors should be thanking us.
Good books are incredibly challenging to write, more so than good software. It's not like you grab Harry Potter and say "I'm just gonna change character names and rephrase some of the text". Most authors recognize that not everyone can afford books and then contend that some amount of sharing is healthy, no different from borrowing books from a local library. If you ask nicely, they will probably send you a PDF for free. But the scale of online book piracy is absolutely staggering and demoralizing, and most of it has nothing to do with taking any serious moral stance. It's just "lol, why pay when you can download for free".
And the siloing of movies by Netflix nd all the other streaming services, and the introduction of advertising into the "reasonable price" tiers, shows that people can and do remember piracy in still an option, for when corporations and copyright holding groups enshittify things. Lately amongst a lot of my friends I've seen more usb stick with movies being shared that even in the heyday of Bittorrent.
I respect copyright, but I can’t respect a lockup period that can push to 130 years or more. For example: if JK Rowling is alive in 4 years the first Harry Potter book will have a valid copyright extending from the 20th, 21st and into the 22nd century. Is it really defensible to say that your great, great (great?) grandchildren should benefit from a government mandated monopoly on your work?
Kind of weird too think about it that way, but food for thought.
If you take away copyright, you reduce the revenue of publishers, which reduces the number of unproven authors they can take chances on. (They're not very good at picking winners from the pool of new author manuscripts, not nearly as good as "agents" like to think they are, but that doesn't matter; they still wouldn't be able to take as many chances.)
100+ year copyrights only help the descendants of the people that have already become extraordinarily wealthy. Cutting off at 2 decades would have close to 0 effect on the huge majority of creatives, while benefitting society immensely. JK Rowling would still be a billionaire, and small authors would be fine too.
While the current copyright length indeed seems excessive, looking at the most successful and determining fairness seems very untoward. As it stands now, an author could build up a library of their writings, and perhaps even sell it for their retirement, as it has value.
Is that what is happening often for indie novelists? I don't know, but to others in this thread know?
I know you were just musing, but I really dislike this "let's create a hyper-invasive tax framework, so that anyone who etches out a little wealth has it capped" concept. There's no wealth to redistribute, not in the way people think. Money isn't real. It's just an indicator of things. In some cases, it's an indicator that someone owns a lot of stock in a company that could be worth 1/20th of its value tomorrow, after a crash, and further is valued at hundreds or thousands of times the actual value of the company today.
In such cases, it's often an indicator of capacity to steer the economy, not of any tangible wealth.
But people need something to strive for. Wealth is one of those things. In a free market economy, wealth is the reward for job well done. It does make people hustle. It's not perfect, but we've seen how poorly and incapable any centrally planned economy seems to be.
As a Canadian, I do believe the government should be involved in certain things. I have a post office, schools, police stations, fire department, food inspections, and so no on that the government is involved in, so I think health care makes sense too. Yet I absolutely do not believe the government should be planning most sectors of the economy, nor should it be meddling too deeply in the wealth ratios of its citizens.
To put this in that context, would you want to have the current US administration determine how every sector of the economy works, and how people are paid, etc, etc, eg central planning? Can you imagine?
Of course there is nuance. Of course there are exceptions. But overall management of individual wealth seems very invasive to me.
If you look back through the annals of Free Software, one often encounters the claim that the GPL was a way to use copyright against itself, and if there were no more copyright, there would be little need for these licenses.
The GPL ones explicitly disallow the removal of user rights from work derived from GPL software. That wouldn’t be possible without copyright.
There needs to be a middle ground, such as: after 15 years of publication any private individual can access and read the work for free, but the rightsholder still controls commercial sales, merchandising, licensing, character rights, movie rights etc.
As opposed to that, books, movies are pirated for personal consumption. Not monetary gains. If someone bought a $30 book, and then ran a BaaS with millions of VC money in his pocket, people in HN would be angry at him, too.
Paradox of tolerance: to maintain a tolerant society one must be intolerant of intolerance.
> Even an open source license misses the point, because the implication is still that one person can dictate how another person can make use of knowledge. It’s still premised on the same dystopian view that a person can own an idea.
Your proposal: to make information free one must restrict how information can be used.
The biggest hole in your argument is just the MIT and BSD licenses. Sure what you say might apply to the GPL and other copyleft licenses - but MIT/BSD are as close to no-strings attached as can be achieved in our current legal framework.
So, clearly plenty of people are happy to freely give up ownership of their works. But others value keeping the code out in the open - not zero restrictions.
This is analogous to the paradox of tolerance, because they both rely on eliminating the variability in what people want when it comes to free knowledge or a tolerant society, and they both rely on using words to create a false dichotomy: tolerant vs intolerant, free vs restrictive. A tolerant society is still tolerant event if it is “intolerant” of murder - just because i used an antonym doesn’t mean those are the same exact concepts being negated.
This discussion is intellectually dishonest. Either some people here genuinely dont understand the concepts of kindness and gratitude, or they do understand them and are just choosing to spread falsehoods anyway.
Just because my beer pub isnt going out of business because you took some free beers doesnt make it ethical for you to exploit my kindness and use those free beers to build your own competing beer pub.
If people are still confused: that setup is not sharing knowledge. It is stealing with nicer branding to help you and your friends sleep at night.
What matter at the end of the day is not what the document pretend about who possess what, but how people feel in their life, what they can access to, and what they are bared to access for which actual reason.
It can't even be purely narrowed on what human people feel like. We all know our species is dependent on many physical phenomena and other species which owe nothing to us.
You can question ownership in the abstract, and I am not even against that conversation. But that does not answer the actual point here. We still live in a world where food, rent, healthcare, clothing, hygiene, servers, tools, and time all cost money.
So if someone gives something away out of kindness, access, public benefit, or community spirit, that does not automatically mean everyone else is entitled to industrialize that kindness for their own business.
Open source is not a mystical anti-property pact.
Open source is not a contract where people are expected to provide endless unpaid labour for others to build businesses on top of. At some point this stops being a discussion about sharing knowledge and becomes a way to justify taking advantage of people’s work.
I just don't like these replies, if this was sarcasm, it might not have worked for me. I just find these social comparisons deeply unserious when the discussions are against theft or harm done to actual people with actual human needs.
We, as "anyone online able to read this", certainly are pushed by social pressure to consider world through that kind of lenses. Not sure money mean anything for any animist tribe isolated in Amazonia.
We can even consider costs, without talking about money. We can for example deem attention span, ecological impact, human relationship, emotional burden. So many things that money tend to weight zero in its process to flatten judgements to dull "well ordered" scalars.
Second of all, “knowledge is free” doesn’t mean it is free to generate. It means that it literally cannot be owned. Property implies scarcity. If I use a thing, it makes it so you can’t use that thing. This does not apply to something like knowledge. The fact that I use your idea doesn’t mean you can no longer make use of that idea. Your analogy is flawed, because drinking a beer means that is a beer that cannot be used by someone else. Intellectual property is an oxymoron. There is nothing actually stopping someone from making use of someone else’s knowledge, and therefore we must artificially construct barriers in order to maximize the ability to profit off of it. In the most charitable sense, this in theory provides financial incentive for people to generate knowledge. In a less charitable sense, it is literal though policing for the sole purpose of extracting money.
I get that making use of knowledge without compensating the person who produced it is “theft” under our system. The problem is not with the act. It is with the system in which knowledge work can only support an individual if these dystopian measures are enforced. Frankly, most knowledge work is built on mountains of previous knowledge work that never is compensated for it anyways, so the model of the world isn’t even internally consistent.
Copyleft was created to protect users of free software from authors/distributors who tried to use copyright to control the software running on the users' computers.
In the case of AI, I don’t think LLM providers are required to make their models completely open source just because they were trained on the data freely available on the internet. Those model weights aren’t the original data anyways, so they aren’t even selling access to that original data. However, there is should be nothing preventing others from creating and selling something produced from those LLMs. Distillation, for example, is free game.
In short, you should be required to make your knowledge freely available. But once someone has access to that knowledge, you can’t stop them from using it however they wish, including if they wish to sell it themselves.
Open source licenses are almost entirely unrelated. They're strictly a hack around the copyright system, and not only that, they literally do nothing other than grant you rights you wouldn't otherwise have. Talking about open source is mostly a distraction. When people say knowledge is free they almost always mean access to knowledge. Open source grants people access and more.
People are not mad that they can't just steal things, they're mad that access to things is tied behind massive gatekeepers (essentially indefinitely...) that essentially exist to continue to enrich themselves while somehow almost none of the money makes it back to the authors, and is sometimes completely untethered from where the money comes from that funds the works to begin with. You can't just freely navigate, search through and consume information, it's all tied up behind various pay walls and monetization schemes while authors starve anyways.
We could have a more equitable and reasonable system that allows broad access to knowledge while providing some approach to monetization that is reasonable for both people seeking it out and people consuming it. There's little point in trying to enumerate the number of ways it could be done. We already have a system for taxes, we already have seen commercial schemes like Spotify, you could slice it thousands of different ways. Plenty of pros and cons. I'm just saying it could be done and we know it could be done.
But it can never work if all media and knowledge dominated by rent seeking gatekeepers standing in the middle whose primary purpose will always be to enrich themselves first and foremost. They will always want to get more and give less, because that is more or less their fiduciary duty.
Therein lies the entire issue. Intellectual property is a paradox and a farce, but our current systems require IP as a concept to exist. 99% of human history existed without the concept of IP law, and yet people still innovated, learned, and shared knowledge. But now we are at a point where human beings can only survive if they have something they can productize and sell to others, and so for knowledge to be produced, it has to turned into something that is artificially scarce, so that those producing it can be compensated for their labor. Maybe this is unavoidable for this point of our evolution. I doubt it, as 99% of human history managed without the concept of IP, and we still produced and shared knowledge.
But even if it is necessary for this stage, it is still a dystopian model. And I imagine we aren’t far off from a point where we can provide for everyone’s basic needs, and therefore we don’t need to introduce artificial scarcity to make knowledge work economically viable.
To be considered open source software, the license cannot impose any restrictions on how the software is used. You are free to use the software for whatever purpose you want.
[0] https://en.wikipedia.org/wiki/OpenWrt (see History section)
For most physical objects and concepts, distribution isn’t a subset of use. e.g. the post office. You can be a user, but you have no ability to distribute the post office - because that’s not a thing that is possible or makes sense. In that case distribution is not a subset of use. Other examples: most saas, e.g. gmail. Good luck distributing gmail.
That would enable the authors, activists and hackers to pursue what's meaningful, instead of the profit of the multinational leeches that do not need to adhere to laws, borders or taxes.
If a zillion dollar corporations (Meta and the likes) can torrent Annas Archive and decimate the copyrights I see it a moral imperative for the people to do the same and spare the pennies that would profit the publishing / media / ... industries to support the authors directly, instead of the trickle up method, where majority goes in to the hands of the dystopican narcissist zillionaire.
>> That would enable the authors, activists and hackers to pursue what's meaningful
I would counsel not using the word "meaningful" in the context of BI. We already have a way of evaluating "meaningful", it's called "money". If society gets to judge what is meaningful or not, well, that's the system we currently have.
BI is about letting people do whatever they like especially if it is meaningless. BI implies an economy (you have to spend the Income on something) so meaningful will always be richly rewarded.
Incidentally publishers exist to act as curators and filters. The value they add is real. There's no shortage of self-published stuff on say Amazon, but 99% of it is drivel. I go into a bookshop to find the 1% that at least someone thinks is worth reading.
Cultural reasons are just a matter of spindoctoring/propaganda.
Nixon almost implemented a form of UBI: https://en.wikipedia.org/wiki/Family_Assistance_Plan
In Alaska there's already a form of UBI: https://en.wikipedia.org/wiki/Alaska_Permanent_Fund
BI by contrast values people over money. It recognises not just the social responsibility of the rich to the poor, but also the dignity of being human.
Some countries are further along that path than others. Health care, education, unemployment benefits, are all steps towards BI. The wealthy are taxed to pay for the poor. Ultimately the suppresses the excess, while raising the floor.
From a cultural point of view, the US has many steps to take before society is really for (real) BI.
That would never happen, in the US. We have a strong “Why do they get special treatment?” thing going on.
I guess we know that Ph.D Research/Teaching Assistants tend to be worked to death, and paid peanuts.
The whiners would be OK with that.
Almost half of US federal tax payers pay 0 income tax.
That seems like a screwy but ultimately more than fair deal for the top 10%.
Especially when the alternative is pitchforks and torches.
> The percentage of workers belonging to a union (or total labor union "density") varies by country. In 2024 it was 9.4% in the United States, compared to 20.1% in 1983, the first year with data suitable for comparison.[2]: 1 [3] There were 14.3 million members in the U.S. in 2022, down from 17.7 million in 1983.[2][3] Union membership in the private sector has fallen to 6.0%, one fifth that of public sector workers, at 33.1% (2022).[2][3] From a global perspective, in 2016 the US had the fifth lowest labor union density of the 36 OECD member nations
https://en.wikipedia.org/wiki/Labor_unions_in_the_United_Sta...
Once meetings and documentation is working, decide on a small issue that you know that more people in the workplace are annoyed by and would be an easy fix for the organisation. Then you write to the relevant boss and present the demand. Either they fix it voluntarily, or you wait a while and then you write to them again and add a demand to have a meeting about the issue.
Escalate slowly and methodically, and when you get the win, lay low for a bit and use it to recruit. Make all decisions by consensus in the beginning, don't start with majority decisions until most or all of the workers in the department have joined. Always escalate slowly and gradually, until you have the power to shut the entire organisation down on a short notice.
Act in a way that makes your union predictable to the bosses. Corporate management is easily confused and you'll prefer them to negotiate over lashing out wildly. When people are inevitably fired, keep in contact with them, both personally and through routines you establish in the union. Some people are unsuited for negotiation, documentation and so on but are good at doing phone calls and checking up on previous members and reporting back.
Focus entirely on things that are important to make the daily work less dangerous, more convenient, more rewarding. Small steps, low hanging fruit first, only one issue at a time. When you have a success, send a note about it to some local labour friendly organisation or publication.
Don't waste time on identity politics, everyone in the workplace who isn't a boss is a worker regardless of what they do or how they're compensated, the union does not need a name or bank account until you have federated with quite a large amount of people. You don't need to register some association to have meetings and make decisions, that's for when you have like a few hundred people and want to throw parties and have conferences and things like renting a place requires you to register with some bureaucracy.
> Cultural reasons are just a matter of spindoctoring/propaganda.
Similarly, there are a number of things that would be incredibly meaningful to all of society (eradication of disease, nuclear fusion, etc) that we choose to deprioritise to instead eat fois gras and fight.
Sorry to jump down your throat on this, we're on the same side, I think BI is inevitable and worthwhile. But it's worth pointing out that BI enables more than just the meaningless things.
We are already spending massive amounts of money on disease, fusion and so on. There's no issue there, and BI doesn't move that needle.
At the moment society (especially in the US) operates on a "add value or starve" basis. (That's an over simplification, but the underlying "morality" us strong in that direction.)
BI moves the needle for those who are not "adding value" (in a materialistic sense.) Artists and Authors are free to spend their time creating works, of which a rounding error will have any value. Sure there's some unappreciated author out there cranking out literature, but there's also everyone else cranking out rubbish.
BI doesn't make 'big things' easier to do. Arguably it makes them harder. Rather it allows individuals to gain satisfaction from little things. Budding poets can write all day long. But if (great) poetry is currently ignored, do not expect much on that front.
I say this not to denigrate BI but rather because allowing the meaningless is precisely its goal. To miss that is to miss the point. It allows people to find worth and dignity without having to add value to society.
I think you’re getting pushback because of this choice of language. It’s not the only goal, but it is a key feature. BI supports choices of how to spend your time and enables freedoms.
I think this is wrong. It's not about value, it's about being submissive.
This is a mechanism that’s very narrow and creates notable distortions between financial and artistic value, just to keep the example specific enough.
I’m also going to take issue with this interpretation of “meaningful” - I’ve known several amazing crafters and artists who have had an incredibly hard time doing their craft simply because the demands of capitalism prevent them from putting the time and effort into honing those skills, finding a market, creating a portfolio, etc.
If anything, I think BI is just as likely to add meaningfully to society as it is to give people the option to do meaningless things.
I am dependend on government not giving subsidies to my competition. I am dependend on them not to raise taxes. I even depend on them for keeping the infrastrukture alive. If I cant trust the state I life in, it is time to move to a different state.
To me it makes you more dependent on the other people in your society - only if you don't earn more than the BI of course.
That is true under Western Capitalism too, you're already dependent on your society, unless you filter and pump your own water; grow your own food; generate your own power; make your own devices; school your own children; do your own healthcare, etc.
For those reliant on BI, instead of greed being the moderator through which your income is determined, it is determined through the democratic processes of state.
Now, if your have an oligarchic dictatorship like USA and Russia, sure you'd need to be particularly worried if you were relying on BI. If you've got a strong democracy (probably true of any country with state-run services that are competitively priced) then BI should work if there's enough total income that it can be redistributed.
I think there are problems with BI though.
It’s worth pointing out that violating a FOSS license is ALWAYS about denying users rights the upstream already gave them. Violating a FOSS license is stealing from your users.
In small doses, and short terms, I might agree with your classification.
But when copyright is 150 years It no longer has anything to do with reward for the author or encourage you creativity, it’s just a cartel.
The knowledge economy is different. It’s hard to see how the system works in a world where everyone has the equivalent of a shoe replicator in their pocket.
Ironically the “free market” only survives by having arbitrary regulations on shoe duplication enforced in the interests of shoe-rights holders.
Free market is just the best way we know of dealing with scarcity, every other alternative that was attempted seems to fail sooner or later. If you invent a way to get an infinite amount of something, that something goes out of the free market rationing.
For the knowledge, a lot of free market more radical defenders don't believe at all in current state of "intelectual property", much less if et has to be enforced by the state. They are for "industial secrets" (if they leak, you lose them), NDAs (if they leak, you enforce the contract) and similar formulas.
If the books were released under an open source license, there would be no problem here?
we grab pitchforks because you violate the licence by making knowledge NOT free?
what exactly is the contradiction?
Well for starters because the research shows that reading on a screen literally makes you retarded.
https://www.nature.com/articles/s41598-023-36256-4
This is only apparent contradiction. The underlying issue is of course the social concentration power.
It's not the same when an author is deprived of virtual money the copyright system entice them to extract from readers regardless of whether they have money or not, and deprive authors from a negotiation mechanism against corporate that swim in money. In both case, with current legal systems the most obvious law enforceable mechanism is copyright. It doesn't mean the underlying issues at stake are the same.
What's really staggering and demoralizing is that humans have all that it takes to feed all mouths, make equal incomes, live in peace in all kind of diversities that encompasses reciprocity of accepting differences, and yet we end up with people dying from war and starvation while other accumulate a toxic level of wealth in a system that tries to uniform everyone and harshly cuts anything that don't fit the standard box.
Not that I think a back door to the shit Disney and other corporate content onwers pull is nessecarily a bad thing. But it is funny to see people here gaslight themselves into believing they have some moral right to just take what others have created.
If many people are immoral, curb the roots of immorality then, fight that one battle - do not shift it to others' sound behaviour exercised with proportion and judgement.
Free/copyleft licences like the GPL are just the way we've been able to effectively make software free in a world in which copyright applies to software.
So I guess we can have a world in which knowledge can be free without starving knowledge creators?
Rowling is a rare exception and very much not the norm.
Where do we do that? All LLMs are still doing it. I've not seen any MIT license or BSD license of all the repos LLMs have gone through. That's excluding copy-left licences. Zero pitch-forks in sight.
> Someone pirates books and it's fine - really, the authors should be thanking us.
I agree here with you. And I'll give a better reason to not pirate for most people. In the seas of endless content, If you want to read/learn from a book - try paying for them. You won't read most books. You'll curate/research better before buying. The books you buy will be worth it in your own mind. You re-consume them more which is very important IMO. For most pirates, I think the issue is not unavailability of resources/content. But need to consume in a better way. Buying slows things down. You'll consume better. Quality over quantity. Even then I have a ton of humble bundle books that I've not touched at all.
Imagine you're a professional writer and it's your main source of income. How would you feel if someone said this to you? Would you still want to write books?
"What an inestimable advantage it would be, if, in every branch of literature, there existed only a few but excellent books! This can never come to pass so long as money is to be made by writing." --Arthur Schopenhauer
The reason I buy books is rarely for knowledge or ideas, its either for a good story in the case of fiction (which the author definitely should have the right to exclusively commercialise), or for the authors explanation of and idea or some knowledge which goes beyond the raw information I could find in the scientific papers or higher level descriptions.
Good storytelling and teaching are valuable and should come with some sort of exclusive rights to control and profit from by the author. And even bad storytelling and teaching should have that same protection from other people distributing it in ways that restrict the authors rights.
Clearly 130 years of protection is insane, and all it does is keeps Micky Mouses lawyers able to buy new yachts. But as others in this discussion have pointed out, after 20 year almost all of the authors who are still earning money off their works are already rich beyond most authors realistic hopes. I'm not sure 20 years is "the right length" for protection, you sometimes hear stories of works being rediscovered and becoming wildly popular more that 20 years past the original publication date (Kate Bush's Running Up That Hill getting back into the charts on the back of it being used in Stranger Things - for example).
There are probably a few thousand of these in the entire United States. The market system heavily under-values writing as is, independent of piracy
Constant payment into perpetuity for the replication of digital information is just a form of rent-seeking. Except, unlike a landlord, you're not obligated to correct defects.
To put it into perspective: even *free* services can be completely inaccessible here if they require payment info for verification purposes.
Acquiring hard currency to spend online is truly a bureaucratic nightmare, and even if you manage it, the annual limit is strictly capped (at around $300).
Some of my published work is pirated heavily. That's not my main income source, so I just shrug and let it go. If anything, I'm probably happy that people are reading my work. Especially if it's people that can't afford it, I'm glad they enjoy my work -- those books got pretty high reader ratings, and it seems to me many readers are actually reading the pirated version.
But I do have friends that depend on this income source, and fighting piracy has become a part of their day job. It's not a fun thing to do, they'd rather spend time working on their next story, but they still have to do this everyday. I feel for them.
Not sure if we will qualify for a bounty, but happy to share! Btw, we are looking for funding from small or large donors who want to help us translate the Renaissance…
I can't quickly tell what all you have archived^, but I have some friends who are academic historians who might be interested in certain categories of work (and could help verify some esoteric languages) - is it possible to search by region or language?
Have you reached out to any types of historians WRT the project? It seems like some PhD students might be able to find some projects in this work etc
^ when I looked at the timeline https://sourcelibrary.org/timeline, I got an error
Please share with historian friends. I’m not great at socials or fundraising but this was really designed to support humanists. It can give DOIs for the versions of the translated books, which means they can be quoted and cited in academic papers.
Tip: Try it in Claude or Claude code (even better)! Just point it towards the source library. It can find quotes and evidence on any topic of interest. Or try the librarian — our source-grounded research agent https://sourcelibrary.org/librarian
Thanks for the feedback, I’ll fix the timeline.
You can add it up!
I have seen Gemini costs change quite a bit when processing very similar books from the same series lately, mainly because thinking tokens have increased about 5x. Has that has happened to you as well?
Edit: for ocr I am using about 15k-25k tokens per page, but I have a complex prompt.
Cloudflare captchas have made the internet unusable for me, and I'm sure it will only get worse over time. I'd much rather just browse (or even torrent) a copy of archive.is or similar. The latter would be much better for privacy, and hey, I run ad blockers anyway.
Well, there is this little conflict of interest
True or not I rather like that one.
This one's good as well. This is an example of a rare good Reddit thread.
Apparently this is the reference: https://www.youtube.com/watch?v=FKCmyiljKo0
But this is what I thought of: https://youtu.be/hdsB99HQ2S8?t=194
All are examples of people with less power protecting each-other from a more powerful force.
But I could be wrong.
I am more surprised to see that there are so few alternatives to it. Or perhaps I am unaware of them but after Facebook and co declared war on libgen, and libgen going down, there were surprisingly few alternatives. Anna was one of the few. I still don't know what happened with libgen, but since the attack it really is kind of semi-gone.
And then you could obviously just buy it physically where buying is definitely owning, so I find that sentence a bit inappropriate for books
[this] appears as a link to a .li address, and that goes bad places.
It won't even open in Brave.
Good thing you have Gemini though.
Or did they lose the domain to scammers and never update the link in the bounty?
You often need an Italian credit card to pay.
The digital world is crazy.
Lying about your assets to avoid paying a lawful fine is criminal. Just because they can’t see your money doesn’t mean they can’t prove that you have it, and can’t jail you for hiding it to get out paying a fine.
financial watchdogs and international treaties make it impossible unless you are perhaps a multi billionaire who can afford to buy people at the political level
And even fewer who are single and childless. (Google would likely go after the estate of anyone who did this.)
This is something they’d want to settle quietly, so the family would have leverage.
The book publishers might actually be the bigger problem. They'd have civil copyright infringement claims with giant statutory damages.
Do you think they care? People with cancer are often laid off from companies. Sometimes a bit after their diagnosis becomes known to the company.
You could look it up and see for yourself. It's applicable to all sorts of collections such as map data and I'd presume also a book database but IANAL so best if you see for yourself - my intention was just to point out (what I hope is) the applicable legal principle for anyone curious about this
I could look up lots and lots of things to see for myself. I wouldn't be reading your comment, and there is not enough time in the universe. Readers don't have time to look up everything other people post. It's up to the commenter, I think, to say what they intend and what's valuable about it.
Humans don't communicate by exchanging links to reference material.
Yeah, but still, I think I'd prefer to do it anonymously than be the legendary archivist rotting away in prison.
The current situation feels untenable with renting. So many regular people I know have learned about VPN, NAS, etc.
Spotify, Netflix, Amazon etc provided OK value for a while, but now enshitification is biting, this is due a massive comeback.
Or something like thanks.dev
It seems like there are some deep pockets funding them.
The problem with using the torrents is they are slow, most of the larger sets have <3 seeders, with many dropping in and out, on home or slow connections. I would imagine other companies have learned from Meta's mistake and don't want to appear in the swarm either, which is why direct access is preferred. 100k for unlimited books access is nothing compared to the other costs these labs incur.
I think this is net positive.
> Purchase all Library of Congress MARC datasets — $3,000 bounty
> English Wikipedia pages about relevant institutions — up to $100 per new page
> Internet Archive Digital Lending — $5000 per 1 million pdf files
> Text version of our full library — $20,000
...
https://software.annas-archive.gl/AnnaArchivist/annas-archiv...
https://chatgpt.com/share/6a4970e8-7fe8-83e9-8f81-3aefd76b6b...
On another note, if Google's cybersecurity were always one rogue employee away from a massive leak, then it wouldn't be Google. What was the last Google leak you remember, defense in depth people.
Surprisingly the order is very specific about DNS registrars and authoritative domains not advertising the AA servers, so sharing IP addresses or alternative domain names through other means, like WikiPedia, is not against the order. Which means that nowadays the Wikipedia page works as a pseudo authoritative DNS.
Training on copyrighted material
--> bad
Actually distributing copyrighted material
--> good
Needless to say, this is backwards. Any copyright holder will be much more worried about the latter.
Another explanation might be a general dislike of big establishments like AI companies and publishers (which glosses over individual authors, but they probably make up a negligible portion of total sales anyway).
Copyright reform is necessary for national security
Piracy helps people who can't afford to pay or have no way of acquiring legally.
LLM training helps megacorps replace people.
If you side with common people against megacorops, you're okay with piracy and against LLM training on copyrighted works.
LibGen is now more or less a dead project. The servers of the original version were reportedly seized a couple of years ago already, and other sites under the LibGen name were notorious for piggybacking the original collection and just plastering it with ads. If one wants to upload stuff, better now to upload it to Z-Lib (not a perfect site, but still) and it will then get picked up by AA in a few months.
LLMs are many things, but one thing they definitely are not is cheap/free to run at scale.
https://annas-archive.gl/blog/ai-copyright.html
> Virtually all major companies building LLMs contacted us to train on our data. Most (but not all!) US-based companies reconsidered once they realized the illegal nature of our work. By contrast, Chinese firms have enthusiastically embraced our collection, apparently untroubled by its legality.
> We have given high-speed access to about 30 companies. Most of them are LLM companies, and some are data brokers, who will resell our collection. Most are Chinese, though we’ve also worked with companies from the US, Europe, Russia, South Korea, and Japan. DeepSeek admitted that an earlier version was trained on part of our collection, though they’re tight-lipped about their latest model (probably also trained on our data though).
It's at least 30 companies, each of which paid hundreds of thousands of dollars.
It really sucks, but I'd rather pirate and know I'm at fault with the artist – maybe I'll buy some tracks off Bandcamp to make up for it – rather than let Spotify cover the transaction with a legal blanket, while the artists get almost nothing in return.
This doesn't strike me as a symptom of a bubble - except in so far as the bubble pushes the competitors models forwards and thus they need to invest more to stay competitive.
They all carry political weights, because humans behind defend their interests, and are promoting some social values.
This answer from Claude is so biased that it is ridiculous
China acts like an entire bloc, not as single companies, and they want to monetize hardware.
ByteDance is going the direct-to-consumer route with their Doubao chatbot (the most popular in China, probably thanks to their social media prowess). iFlyTek seems to be angling for enterprise and government use cases, where they already have an in.
The companies that have released weights have in common that they didn't have a monetization channel lined up and their models weren't good enough to make people pay attention with just API access. (You can see with Qwen Max that the calculus can change towards not releasing weights for better models.)
And who exactly among the investors is having their complement commoditized? When Nvidia releases Nemotron, the story is clear, but it's less obvious for say Z.ai's GLM.
> I have never said they always act as a bloc
Pick one
Fundamentally, you can never distill your way to being the teacher, so these approaches will not advance the frontier.
[edit, after thinking about it I think my phrasing is unfair. It's not necessarily that aren't able to do it, but they haven't yet shown that they are willing to do it.]
/s
Not yet.
If there is a need someone will come and fulfill. Personally for me now I do not even want to use top models. Professionally I use AI to help with the coding using Junie agent that comes with IDEs from JetBrains. Junie is told to use Gemini Flash and works fine for what I ("I" being an emphasis here) ask it to do. I tried more advanced models and different vendors only to discover credits going down the toilet without any extra benefit.
Are you sure?
What if you distill from 10 teachers?
Consider Cisco. On the 31st of March, 2000, it was valued at US$77.31 / share, which in inflation adjusted terms is $150.46 (above the current price over 26 years later). This valuation was on the basis of speculation that the price would continue to go up and Cisco would get a large cut of the industry profits. Cisco's business is still valuable, it was just treated as overvalued by the market.
Similarly, if we go back to one of the classic examples of a bubble - consider Dutch tulip prices in 1636; speculation drove future contracts high. But tulips still have value to people today, it's just the price was higher than was sustainable.
Lots of companies will pick them up for scrap metal prices and host them for fractions of what we are paying today.
That's the nature of bubbles.
(a) work out how Google books exposes fragments of books, and see if there's a systematic way of using this to get whole books. For example, a naive approach might be to find any fragment of the book by searching some exact phrase. Then, you can search for an exact phrase from the start or end of the fragment it gave you, hoping it will show you the previous or next part of the book. You can then just loop that to get the whole book.
(b) once you have (a), you need a way of bypassing Google's bot detection/rate limiting. I don't know what current state of the art is, but there may be a solution for sale out there. E.g. you pay to receive a cookie or browser state, and use that to fetch the URLs from (a). Or if you're good/already in the scene, you could do this part yourself.Between all the piracy, and all the AI training and the purchase/visitor-circumventing AI services, the practice of writing and publishing genuinely good work is being wiped out.
We're killing the goose that lays the eggs, for selfish gain.
In such a world, isnt it useful that governments are stupid enough to give adversaries reasons to undermine it? When the government props up a corporate tyranny domestically, and racketeering, should we make a temporary alliance with all its enemies?
(Eg., the provision to AI companies of all corporate secretes and competitive practices via prompts, eventually to be used against their capital interests and their labour interests).
Even if projects like AA didn’t have nation-level support, academics would find a way to keep as much of it as possible going. After all, we’re the ones who compiled the bulk of pre-2020 material, and we’re the ones who do all the hard work of scanning from our institutional libraries stuff that doesn’t exist anywhere in digital form.
We already did that when the internet collectively agreed decades ago that everything digital should be free for anyone.
We're now 20 years downstream of ad-blocking being a virtuous good, and piracy being the ultimate show of liberty, and now suddenly everyone cares about the creator's revenue stream.
The mask slipped and unsurprisingly the internet is a bunch of selfish morally stunted children. Some of them even pushing 50 years old.
Yes, I am talking to you with the 4TB of pirated content, proud of not loading any ads in the last 15 years, and getting enraged over LLM training.
That's oddly-specific :-)
In any case, I have no pirated content that I know off, neither proud nor ashamed of blocking ads[1], but I still get annoyed that a bunch of VCs can use their invested-into companies to launder all the worlds IP, then sell it back to them.
[1] Who feels proud of blocking ads? It's like feeling proud of tying your shoelaces: "Good job, well done, but that's the expectation, son".
Most of the best literature in the English language was written before modern IP law was even a thing. There's very little good literature written by authors primarily motivated by money.
> best literature
What does that even mean?
But there is also some consensus. For music it would be Beethoven, Mozart, the Beatles, etc.
For literature it would include Tolstoy, Shakespeare, Cervantes, Dickens, Austen, Tolkien, and many more. I would bet it will eventually include Stephen King but it's too early to make that call now.
Not to mention, many people would disregard your answer for putting Stephen King (and Tolkien, to a lesser extent) in the same sentence as Tolstoy and Shakespeare.
There is no "better" anything when we talk about culture. No consensus, etc. There are just opinions.
I can only think of one writer off hand who wasn’t a wealthy landowner, although it is a particularly notable example; that of William Shakespeare.
Shakespeare wasn’t poor (his parents seem to be of upper middle class standing), he was able to get a basic (but not a university) education and then pursue an acting career (with perhaps a side hustle as a teacher). Whatever the case he certainly wasn’t independently wealthy before he started writing, he needed to earn a living.
He did seem to be in it for the money (and fame) since he wasn’t just a writer he was an actor, theatre owner, and something of a celebrity, and he did make enough money to become a wealthy landowner by the time he died.
I’m not sure piracy or AI training are really affecting book publishing dramatically. But if you have data, I’d be curious to see it. AI scraper bots are a total pain for online publishers and FOSS sites, but AFAIK they’re not really harming book publishing directly.
The consolidation of publishers and Amazon’s own practices are probably worse for authors than “piracy”.
What you can do is make sure people can pay you easily, and not put (a lot of) hurdles in your readers way. And when people can’t afford to pay... maybe let them enjoy your work still, and you’ll get a couple more loyal fans who would pay you when they’re able to.
At least this was my world view before AI has arrived and ruined^W disrupted everything. Now I’m not so sure.
If you shouldn't be able to copyright GRAPES...you shouldn't be able to copyright BOOKS.As far as what I ought to be able to do with the thing? Beat it. Of course there are exceptions. You say, "Don't scan the book and upload it to object storage and place it behind a CDN so Ahmed from Tounis can access it". I say, "Don't make wine".
Poetic license, pal!