This might actually be a good use case for a blockchain. Hashing the data that's added to the archive and then putting the hash in the blockchain would reasonably prove the data in the archive hasn't been modified at a later date.
It’s too bad all you can build with that is a meager, profitable SaaS business, not a wild speculative crypto-billionaire rocket ride.
https://nrc-cnrc.explorecatena.com/en/
Using a company called Catena:
Chunk-based hashing could work. Corresponding those to document structure (paragraphs, sentences, chapters, ...) might make more sense.
I'm familiar with RDA, FRBR, and WEMI, somewhat.
https://en.wikipedia.org/wiki/Functional_Requirements_for_Bi...
Here have yourself a timestamp https://petertodd.org/2016/opentimestamps-announcement
https://github.com/opentimestamps/opentimestamps-client
pip3 install opentimestamps-client
ots stamp myfile.txtUnfortunately, the files themselves aren't public, and each file contains dumps from hundreds of websites, so even if they were public they're not the easiest thing to verify.
Still, being the guy behind OpenTimestamps I should point out that in this case I don't think timestamp proofs really add that much: Reid's claims seem dubious even without cryptographic proof.
Or they could just implement trusted timestamping (RFC 3161). Using a blockchain is a heavy-weight solution and is rarely the right one.
https://en.wikipedia.org/wiki/Trusted_timestamping#Trusted_(...
You really need better auditing than that, which is why the certificate authority infrastructure now relies on a blockchain - Certificate Transparency - for auditing. Similarly, for timestamping specifically, Guardtime has used a blockchain for auditing their timestamps since well before blockchains got called blockchains.
Surely if content is served over HTTPS with a valid certificate, it should be possible to save (possibly as part of a WARC) a "signature" of the TCP stream that would go beyond proving that a web archive was created at a certain time, but also that it was served using that person's private key and thus from that person's web server. To claim otherwise, the subject would have to claim that a fraudulent certificate was generated for their domain or that their web server was broken into.
Basically, the way the crypto math works in HTTPS is it's a symmetrical proof that only proves that either the sender or the receiver sent the TCP stream. Normally that's OK, because you trust yourself. But in this case the problem you're trying to solve is to prove what happened to a third party who doesn't trust the receiver, so your idea doesn't work.
The certificate is used to sign (parts of) the values used to create the master secret. It doesn't sign anything after that.
1. The web server doesn't sign content; HTTPS keys just provide encryption.
2. The HTTPS connection operates under the web server's certificate; it'll serve any file without regard to who created it.
If you're not trying to get rich quick, however, something a Merkle tree is a great fit and it seems like there'd be value in a distributed system where trusted peers can vouch for either having seen the same content (even if they cannot distribute it due to copyright) or confirm that they saw you present a given object as having a certain hash at a specific time. Whether that's called a blockchain is a philosophical question but I think it'd be a good step up over self-publishing hashes since it'd avoid the need for people to know in advance what they'd like to archive.
To make that concrete, imagine if the web archiving space had some sort of distributed signature system like that. The first time the integrity of the Internet Archive is called into question, anyone on the internet who cared could check and see a provenance record something like this:
IA: URL x had SHA-512 y at time z
Library of Congress: URL x also had SHA-512 y at time [roughly z]
British Library: We didn't capture URL x around time z but we cross-signed the IA and LC manifests shortly after they crawled them and saw SHA-512 y
J. Random Volunteer Archivist: I also saw IA present that hash at that time
That'd give a high degree of confidence in many cases since these are automated systems and it'd lower the window where someone could modify data without getting caught, similar to how someone might be able to rewrite Git history without being noticed but only until someone else fetches the same repo.
(Disclaimer: I work at LC but not on web archiving and this comment is just my personal opinion on my own time)
That's what makes the blockchain useful - to change anything you'd need to regenerate all the hashes after the point you want to modify. That's a lot more difficult. Having a proof that's generated by network of parties (like a cryptocurrency) would add to the trust level, but it's not essential.
EDIT: If the archive published hashes of everything they added daily in the NYT (or any publication) it would become unprintably large. It would only work digitally, at which point we're back to something that's trivial to modify...
(If you could get the same hash then even a block chain won't give you integrity.)
If your list of hashes is huge, you can just print the hash of the entire list. There's no such thing as unprintably large for mere attestation.
I suspect the hardest part of doing that would be simply that you don't fit into their pre-existing categories.
FWIW, if you plan to do that, I'd suggest you put a Bitcoin block hash in the NY Times instead, which would prove the timestamps of everything that's been timestamped via Bitcoin. You can then timestamp your own stuff for free via OpenTimestamps, at which point your proof goes <your data> -> OpenTimestamps -> Bitcoin -> NY Times.
Timestamps are additive security, so it makes sense to publish them wisely. But if you're going to do that, might as well strengthen the security of as much stuff as possible in one go.
You can put any classified ad in any category you want; the newspapers don't care.
I proposed to my wife by placing an ad in the real estate ads of the Vancouver Sun because I knew she'd see it there.
A very similar example is found in git repos: while normally you'd have every single bit of data that lead up to git HEAD, you can use git in "shallow" mode, which only has a subset of that data. If you delete all but the shallow checkouts, the missing data will be gone forever. The missing data is still protected from being modified by the hashing that Git does - and you're guaranteed to know that data is in fact missing - but that cryptography doesn't magically make the data actually accessible.
Kind of. The current state of the archive is mutable, but that changes to that state are logged to an append-only edit history — it's that edit history that is the "blockchain", and starting from a known good state and replaying all those edits must produce the current state. In fact, this is how cryptocurrencies work too — the state is the balances/utxo set, and the blockchain records transactions, which are effectively just mutations on that state.
In this situation, you'd look at the current state and find the deleted snapshot missing, but the edit log would have an entry saying the snapshot was added (and what its hash was at the time), then another entry saying it was deleted.
I believe this would also be an issue for things like Filecoin/IPFS but I’m not sure if the liability issues are different or nuanced.
[0] https://www.theregister.co.uk/2018/03/19/ability_to_dump_ill...
It's not as robust as a blockchain (maybe!) but it's easy and I've been doing it a good bit longer than 'blockchain' has been talked about. More importantly, I can use it to prove that I possessed certain files at certain times, historically.
> I can use it to prove that I possessed certain files at certain times, historically.
Your twitter account would no longer be anonymous at that point. What's the utility of it being anonymous now?
But you can't force a publisher to use something like that, especially if it's the publisher that wants to deny its authenticity.