Hacker News Reader

Comment by myself248 | Hacker News Reader

I can't understand why there isn't inertia behind restarting ia.bak or similar efforts. That seems critical to me, way more so than an official mirror which would be vulnerable to the same takedown and other pressure.

View on HN

Around 1 PB of the Archive is available via torrent for backup. The full Archive was around 10 PB back in 2012 so I imagine it's quite a bit bigger, now.

https://help.archive.org/help/archive-bittorrents/

North of 40PB nowadays. You can find these stats buried in their public links. They do not hold up well to traffic so I will not link them.

Our numbers as of the end of 2021: https://archive.org/web/petabox.php

    4 data centers, 745 nodes, 28,000 spinning disks
    Wayback Machine: 57 PetaBytes
    Books/Music/Video Collections: 42 PetaBytes
    Unique data: 99 PetaBytes
    Total used storage: 212 PetaBytes

Geeze, Its much higher then I though! 212PB is damn hard to store!

I can understand why.

Because it’s a lot of work.

I'd also guess because (potential) mechanisms to do so would be attacked on forums like HN because they sound too close to crypto/web3 and most people wouldn't bother to separate ideology from technology and create a PR nightmare for internet archive. So it's probably harder to recruit people to work on it.

See Filecoin Foundation and their work with IA.

Just do something similar to folding coin, which is used for things like protein folding. Perhaps an Internet archive coin that can be earned by hoarding backups of the archive on hard drives.

Its also a really hard issue. My fav is IPFS, Its the most common suggestion but it only works on paper. IPFS Allows people to pin things, but it still has to be stored /somewhere/ and, it gets really damn slow once you throw more then a million files at it (I know... we tried)

Overall, Its just a giant archive, its hard to store that much data in a cheap manner that will last 20 years.

Reply on news.ycombinator.com