Hacker News Reader

Comment by ignitionmonkey | Hacker News Reader

We can't and shouldn't expect people to keep their old domains forever. We need a way for pages to be signed and hyperlinks to enforce authorship. When we link to stuff, we should have a way to say whose stuff we're linking to. It's no different from installing signed software and using trusted repositories.

This is one of the reasons I created a proof-of-concept web extension that verifies links and pages using PGP. On a mismatch, it flags the page and offers a web archive link instead.

https://webverify.jahed.dev/

It was pretty fun to make, but currently due to performance, Web Extensions API doesn't provide the features to do this perfectly. Firefox provides just about enough additional APIs to hack it together.

View on HN

One idea that's been around for awhile is to identify files by their hash. That has pros and cons. The good side of that is that the file is immutable; you can't accidentally link to something else unless someone can manufacture a hash collision somehow. The down side is that if the file is corrected in some way, you don't get the fixes.

In a lot of the peer-to-peer distributed hash table designs, all you need to retrieve a file is its hash.

https://en.wikipedia.org/wiki/Content_addressable_network

Problem is, you have to download the entire video to check the hash. That's not how video embedding works; the client browser is just handed some link, and it obtains pieces of the video, rendering it instantly.

Basically, little segments of the video have to have a signature which is continuously validated. Or something like that.

I don't have expertise in video codecs or file formats, but couldn't you hash the first N bytes of a stream? Stream those N bytes to the client and if it matches start the video, else stop the download and not start the video.

Presumably for SHA256 you only need to hash ~256 bits; what's anyone gonna do, try all possible combinations to find a collision?

256 bits is only 32 bytes, and most file formats have standard stuff right at the start. Collisions would be very common.

If you only hash/check the first N bytes of the video stream, the remainder of the video could be anything.

They will keep the first few seconds or minutes of the original video, bit-exact, and then switch to porn. The player needs to validate every section.

This has a number of problems.

The most egregious is if I'm an attacker and I have the file you request I can hash the appropriate portion you'd use to verify it but fill the rest with junk or exploits. You'd receive the file, it would emit the correct hash, yet be not what you were expecting.

For video especially what you receive isn't necessarily predictable by the client. With HLS or MPEG DASH streaming the video you receive could be one of a number of different encoding e.g. lower or higher bitrates to deal with changing network conditions. The actual m3u8/mpd file you might receive could change arbitrarily as the video provider adds or drops different encodings. The hash of such a file today isn't guaranteed to match the hash tomorrow for entirely banal non-malicious reasons.

Fun fact: the UUHash algorithm used by the FastTrack network (Kazaa, Morpheus, etc) only hashed the first bit of a file. Hashing a large file took forever on hardware of the day. Even hashing small files was non-trivial. The RIAA through various fronts would insert spoofed files where the first portion of the file was legitimate but the content of the file would be junk or annoying sounds. The files would be named like any other MP3 someone was searching for and even have seemingly good IDv3 tags.

> Fun fact: the UUHash algorithm used by the FastTrack network (Kazaa, Morpheus, etc) only hashed the first bit of a file.

The first 300KiB plus a series of 300KiB chunks at exponentially-increasing offsets, per Wikipedia. But still a small fraction of thw file.

Well explained. Thank you!

Merkle trees basically accomplish this with separate hashes over every N bytes, so that the content can be verified continuously as it's downloaded.

Assuming there is no malicious intent behind the embeds you could have the hash in the header of the video.

I think that in the context of the topic of this submission "replacement by porn" is being regarded as malicious intent.

That's not really a problem. You don't hash the entire video, but do something resembling a merkle tree. i.e. look at a torrent, they're identified by a hash but you can download and verify a random chunk

Right, merkle tree! OK, so the embedding site only stores a single hash: the root one. This hashes the the remaining hashes. The first thing we fetch from the video is those hashes and if their hash doesn't match, we flag/ignore the video and refuse to play.

Multiple levels of the tree can be stored throughout the video file. The first level after the root can be for major sections, like 5 minute segments. The next levels are then at the start of each 5 minute segment, giving hashes for one second chunks.

If the root hash checks out, we get the 5-min hashes. If they check out, we get the hashes for the first 5-min block, and if those hashes check out, we start to play the video, validating every second of it against a one second hash from the 5 min block. Then we get the next 5-min hash block and so on.

Kind of thing.

In actual HLS streaming videos are served in 2-10 second segments, a size small enough that you realistically can hash each segment and verify each segment. Youd have to implement it as an extension of the HLS protocol, so probably as m3u8 with additional fields, maybe as comments.

That's already how it works in IPFS or Freenet. The hash is not for the data itself, but for a metadata document that contains a list of hashes to the data broken up into smaller blocks.

Sounds like something subresource integrity[1] could be expanded to include.

[1]: https://developer.mozilla.org/en-US/docs/Web/Security/Subres...

This is were a signature would come in handy, if you want the resource to be updated. Just keep track that it was signed by the same private key.

I had a personal project that I got bored with so I let the domain expire. Then I got emails from former users saying that the domain was now hosting malware. So yeah I would like it for all the old links to the site to somehow know that the owner has changed. Not sure what a reasonable solution would look like though.

One solution is to have a unique URI per file that is independent of DNS. Decentralized file storage (e.g. FileCoin / IPFS) might serve this purpose...

Definitely but I think we need something that will work with the web we currently have while these bigger ideas are fleshed out and adopted.

Also, while IPNS covers the issue of linking to dynamic content, it's worth mentioning IPFS will have similar issues with DNS as DNSLink and similar domain-driven solutions are used to cover its usability issues (long, random URIs).

I think long, random URIs are fine for embedded content, actually. Most embeds are short, random URIs prepended with a trendy domain name.

If you could "permalink" certain content for embeds, that'd probably solve the issues, right?

IPFS works pretty well in the style of progressive-enhancement with the existing web for static content specifically. If you want to link to a resource that's available through IPFS, then you make the link point to an IPFS gateway that you trust and expect to stay online (possibly your own on your own domain), like https://example.com/ipfs/Qm_IPFS_HASH_HERE/. Anyone that has a browser with direct IPFS support (either because they're using an IPFS extension or they're using a browser with built-in support, which might get more popular if IPFS takes off) will have their browser recognize the URL format and just fetch the file by hash directly from IPFS, and it won't matter if example.com is still up and serving the file. For everyone else, the link will work as long as example.com is still up and acting as an IPFS gateway. If example.com ever goes down, then users can make the link work by installing an IPFS extension or manually replacing the example.com domain in the link with the domain of any still-active IPFS gateway, and any admin in control of the page could fix the link similarly.

Could SRI be used for this? [1]

[1] - https://developer.mozilla.org/en-US/docs/Web/Security/Subres...

SRI/hashing works for static content. Though it's worth mentioning it's a SUB-resource feature (images, scripts, etc.). It doesn't work for hyperlinks to other pages. Even if it did, it's a different use case.

Say I link to an article by Author A that has comments in it (or even a footer, relative timestamp, sidebar, etc.). Hashing won't work as the page is always changing. I want the link to always go to Author A but I don't care if the content changes. That's the sort of use case signed webpages and hyperlinks with enforced authorship covers. It's less about what's on the page and more about who created it.

Good points. I would guess that for something to be implemented, it would have to be easy for browsers and API tools to check once per domain and cache the response and should probably be something that already exists and has been adopted. Maybe a page could have a meta tag or header that contains a hash of the destination sites DANE signature? Something like "targetref:somedomain.tld expectsig:39726a2fe2bb052cf00e6b95a8385f7" based on tools like danecheck [1] or maybe DNSSEC but that is very poorly adopted.

[1] - https://github.com/vdukhovni/danecheck

Finding a way to embed the domain registration date might be sufficient, that would cover most of the expiry situation

There was a ietf or similar registry that used your domain and registration date to carve out your namespace, ie dns.2021.07.26.com.example would be your prefix. Pretty robust. Can’t remember what it was anymore

The solution I was going for with WebVerify is more web-centric rather than domain-driven, which I think is a better fit for webpages. It can be enforced at the hyperlink-level for shared domains (like GitHub Pages, University web spaces) and works for static resources without needing to configure external resources. The only really complicated part is PGP but that can be solved with better tooling (as seen with Keybase).

Could URL authorship confirmation be implemented on top of TLS? If someone takes over a domain, the final certificate in the chain will be issued to another entity, and that could be enough to trigger a notice. Could be achievable with a centralized registry/crawler like Internet Archive, but one that only keeps track of domain:certificate mapping.

Of course, the devil’s in the details (infrastructure/organizational changes can trigger false positives; shared hosting setup can cause false negatives; it presumes that if the original entity abandons a domain they’d revoke the cert; etc.), but IMO it wouldn’t be worthless as it is.

Most certificates (including all free ones) are domain validated only so the entity being certified is just the domain and will not change with a new owner.

If you’re talking about shared hosting, yes, it wouldn’t be covered by this model.

But other than that, if I have a cert fo xyz.com, and I abandon the domain, even if you buy it you’ll be forced to issue another cert for it.

If it was recorded somewhere that xyz.com = my cert, it could serve as a mechanism to verify that given URL is at least is supposed to be under my control and a warning could be shown if another certificate is being served now.

Kind of like HPKP, but with longer lifetime (longer than domain name registration term) and a centralized registry tracking certificate:domain mappings rather than each individual user agent cache.

Obviously, no one would adopt it due to being a devops nightmare.

Right, you can pin the certificate or the public key in it - but that's much more specific than the entity that the cert is issued to and as you correctly noted is not practical for browsers to do automatically since keys and certs do get rotated without a change in owner.

It's not a solvable problem.

Domains need to be short to be memorized, which makes them scare and valuable.

And having two forms of URLs is undesirable; just look at AMP.

Also, Signed Exchanges for CDNs.

https://developers.google.com/web/updates/2018/11/signed-exc...

Reply on news.ycombinator.com