One day left to help Internet Archive reach its donation goal | Hacker News Reader

One day left to help Internet Archive reach its donation goal

archive.org

220 pointsstriking42 comments

I really dig the basic idea of the IA and I'm an avid user of the site. But, it kind of scares me that they in one sentence say that they will "make our cultural treasures accessible to everyone. Forever." and in the next one asks for money to make it happen. "Forever" is a bold statement and I can't be the only one that sees a contradiction in this. I guess what they're saying is: if the money runs out the data will disappear.

I've worked closely with archivists and preservationists in the cultural heritage sector and I know that to keep the sacred promise of saving something for an eternity you have to have strategies for what you save in the first place. However, I don't see a clear strategy with the IA. The Internet Archive (not the Wayback Machine part of it) is basically a site where I can upload whatever digital litter I like as long as it's not protected by copyright and they (without questions asked) store it "forever". It's fantastic in a sense. But is it sustainable in the long run?

For the the Wayback Machine I can force a snapshot of a site whenever I like and as often as I please. I stumbled across the snapshot for this page today [ https://web.archive.org/web/20140415000000*/http://www.econo... ] and I think it illustrates what I'm trying to say. During march 2014 there was up to 4 snapshots a day, while august did not have a single snapshot. I guess people where trigger happy with the "Save Page Now" [ https://archive.org/web/ ] feature when the article got publicity. Could it perhaps be a better idea to snapshot at certain times and find smart algos that can detect important changes in the page?

I'd love to hear some replies to my criticism. I say what I say because I love the IA and I hope it's being taken care of in a way that will truly make it be there forever.

It would work if they have capital to the point where they can run the site on the interest/dividends paid out on investing that capital. I highly doubt they are in that position but there is a scheme where that will work.

That would likely require a very substantial amount of capital. I too doubt they are in such a position.

List of sponsors: http://archive.org/about/credits.php

Made me very happy to see Library of Congress there.

Until the next 2008. It seems like a financial crisis would put that plan into a tailspin. And there have been a bunch of those in my lifetime. I wonder if there's anything with more permanence?

2008 was a non-event in those terms. I'm not much of a stock buff but if you're going to be in for the long haul you should evaluate the stock market over the long haul.

Well, no. In 2008 the endowment payout would be much lower. The foundation would have to choose to grant a significant amount of the principle to the operating fund.

That's the same as every donor having investments and just paying the site some dividends instead. In fact, I would rather have $1000 and pay the site $10 a year than give the site $1000 and trust it invests it wisely.

"if the money runs out the data will disappear."

I wouldn't expect anything else - data storage is cheap these days but not free.

The same can happen when physical archives (private and/or state run) are not properly maintained because of lack of funds.

Can IA be mirrored like Wikipedia?

It seems like there might be all kinds of copyright problems when mirroring IA. With Wikipedia, everything is licensed under Creative Commons. But IA is a copy of all kinds of websites with all kinds of licenses. It's a big effort for them simply to remove content that companies rightfully request be removed.

Much (all?) of IA content already has metadata for licensing, e.g. out-of-copyright books. It should be legally ok to mirror the CC and public domain content, but I don't know if there's an easy way to extract a dump or torrent.

Check out robots.txt -- most companies use it (in this case) because it's easier than dmca takedowns.

"make our cultural treasures accessible to everyone. Forever."

They're stating their mission and then asking for donations. At least that's how I see it. There is no endeavor that is guaranteed to succeed. Money isn't the only problem they are facing.

I personally greatly prefer that statement to something like "if the money runs out, we'll have to close IA. Please donate." which somehow sounds like blackmailing. EDIT: The former doesn't feel like marketing to me, might lead more people to donate and I would expect most of us (like you and me) to understand that they do not make any promises, except doing their best.

Yes, compare with recent discussion about Wikipedia fundraising language, http://www.quora.com/Are-donations-instead-of-ads-or-chargin...

"The Wikimedia Foundation recently raised $20 million. Assuming a generous $3 million to keep the projects online per year, that's over six _years_ that the projects could continue operating before needing to ask for money again. Contrast with e-mails and in-site donation advertising that suggest that the lights will go off soon if readers don't donate today."

>It's fantastic in a sense. But is it sustainable in the long run?

I don't worry about it that much, when that becomes a problem then a solution can be investigated. In the meantime I extract material benefit from archive.org on a regular basis so I donate.

More worrying (to me, anyway) than the potential to run out of money is the Wayback Machine's policy of permanent, retroactive exclusions.

A copyright holder, rogue admin or anyone who buys a used domain can immediately destroy the entire history of a URL they control. This isn't "accessible to everyone" and it definitely isn't "forever."

I understand that rights holders, etc need a certain amount of protection but this policy hamstrings the project. If content can be removed, the archive fundamentally can't be trusted.

It's like the perpetual "going out of business" sale.

The issue is that any free service needs a revenue model to stay afloat, and folks like wikipedia will turn their site into a PBS pledge-break begging platform when they don't actually need the cash. (Ivy league universities always cut back budgets and layoff people in bad economic times, even when the endowment can fund the school indefinitely because keeping donations coming and being seen as "doing their part (even though it's unnecessary)" are the primary goals.)

PS: Does anyone know IA's financial position?

> But, it kind of scares me that they in one sentence say that they will "make our cultural treasures accessible to everyone. Forever." and in the next one asks for money to make it happen. "Forever" is a bold statement and I can't be the only one that sees a contradiction in this. I guess what they're saying is: if the money runs out the data will disappear.

It's not really a contradiction. It's simply an indication that, to reach their goal, they need financial support. In this sense, it's no different from any other organization - no matter what your storage model is, it's never going to be truly free in the end, so the money will have to come from somewhere.

> I've worked closely with archivists and preservationists in the cultural heritage sector and I know that to keep the sacred promise of saving something for an eternity you have to have strategies for what you save in the first place.

I'd like to see a well-argued source on this. Selective saving is exactly why old archives are often so incomplete. It seems to me like solving a problem that doesn't exist, and with severe consequences.

The Archive is trying to be a sort of modern library of Alexandria, and in today's society, that means taking in content from anybody and everybody. Creation of culture and information has become significantly more decentralized than it has been historically, and archives need to adapt to this new reality.

Done. I also didn't hear about this until now, but surely the 2-to-1 matching is very nice.

We really need to donate to this kinds of projects. We can't let our technological past be forgotten, I think.

Didn't hear about this until now. Just donated because I want archive.org to life a long time. The do an important and unique service for the entire web.

I'm glad they accept donations in bitcoins (not converted). I've been donating some of my early-mined bitcoins to charities. I tend to be more generous with them than I could be with a straight credit card payment. There must be a psychological effect at play there.

https://archive.org/donate/bitcoin.php

Done. archive.org is a shining jewel on the Internet. In many ways it's the fulfillment of the promise the Internet once held of globally accessible information.

A supporter will match 2-1, so your 10 bucks becomes 30 bucks :)

I agree this is a great cause, but did I miss why there is only 1 day to go until a goal is reached? They don't explain the consequences of only getting USD1.48 million instead of 1.5.

There is someone who said (I'm sorry I can't find the quote) that they don't donate to charities who don't specify an aim for a campaign or how they will know if they've succeeded. I like the spirit of this and feel that more information is better than just "Come on, guys, we need 1.5mil before 12am!".

Does archive.org let you download dumps of all of their data? If they make a weekly torrent containing their whole data-set it would be easily immortalised...

They hit 10 petabytes two years ago* - which is 10000 terabytes. Torrents don't really cut it at that scale.

* http://blog.archive.org/2012/10/26/10000000000000000-bytes-a...

> They hit 10 petabytes two years ago* - which is 10000 terabytes. Torrents don't really cut it at that scale.

I see no reason why torrents couldn't handle this. Data transport through bittorrent still works even with large numbers of bytes.

You could use floppies to transfer terabytes of data, but that doesn't make it a good idea.

At these scales, bandwidth to disks isn't trivial - at a gigabyte a second, it would still take over 130 days to build the checksums for the torrent. I don't think Bittorrent fundamentally allows multithreaded torrent generation.

I think they bundle their data into WARCs of a few (dozen?) megabytes. Copying those around is much more straightforward, with much lower latency.

I think he could have meant that some DHT could have been used, it needn't be specifically torrents. You could then volunteer your machine to redundantly store, say, 5GB of the archive.

If you are willing to spend time copying files around you should donate: your pay-per-hour probably far exceeds the time it would take you to dilly dally with billions of WARCs. Even if you flip burgers for a living. There are far better ways to contribute than by typing 'cp'.

A good portion of the images/video are spam - take that into account.

Who is this supporter?

To elaborate - if I am going to spend my money, I want to know who is benefiting from it.

In this campaign, some not specified contributor will have 2 times more (moral) influence than all smaller contributors combined.

Somebody will get its dose of feel good and I want to know who it is.

I really don't understand how they can be fine with so much illegal content hosted on the site.

Ex: https://archive.org/details/JohnMayerMusic

This is being featured on the homepage! No way this is in the public domain.

Did you read the detailed Rights section on the home page?

Of course not, do most people?

What are you referring to, that they will take things down only when a complaint is filed? That doesn't really speak to the quality of the archive that it's there in the first place.

It's pretty hard to miss. On the link you provided, it's a great big section right below the top one which explains which John Mayer recordings are not allowed on archive.org and which are.

More broadly, archive.org does operate in a legally gray area. But by trying to be a comprehensive archive but providing straightforward mechanisms for rights holders to opt out (including retroactively)--and by being a non-profit that doesn't run advertising--they seem to strike a reasonable balance for most people.

I see, FYI this notice is nowhere in the new beta site. I see it in the original.

Perhaps most people do not, but one would expect that those making assumptions about the legality, or lack of, of the content would have a look

> That doesn't really speak to the quality of the archive that it's there in the first place.

The quality of the archive? Not archiving something because of copyright is what negatively affects the quality of an archive.

Reply on news.ycombinator.com