Hacker News Reader

Comment by jabbany | Hacker News Reader

I keep imagining the fantasy scenario where some day I stumble on a software download or repo or something. Nothing special, no fanfare, just a random piece of code to do some random task.

Inside the repo or zip, is a simple text manifest file. The file has some bog standard readme stuff and then lists all the files and their hashes, you know, so you can check nothing's corrupted whatnot.

But in that list of files, is a line item for the manifest file itself and along with its own hash! Something that on the surface looks completely innocuous but becomes profoundly impressive as you ponder it.

Kind of a low-key [fridge brilliance](https://tvtropes.org/pmwiki/pmwiki.php/Main/FridgeBrilliance) kind of flex.

[Bonus] I'm also reminded of this paper: https://vision.cornell.edu/se3/wp-content/uploads/2014/09/ge...

View on HN

Sounds like the basilisk collection: https://suricrasia.online/unfiction/basilisk/

In reading that page, I was super taken in and immediately wanted to get a copy of the collection; trying to follow the links left me sad to realize it's a fictional story, but it's an amazing piece of fiction none the less.

Yeah, it reminded me of the Three-Body Problem... "hard" science fiction that's believable as an extension of what we currently know

That would be a great, and terrifying at the same time, first contact response message to us.

proof-of-superiority if you will.

One kind of that was when DRM rolled out... my memory is fuzzy on what this is as it's a vague recollection of an article once read and I hope someone will recall it and link it.

But in the cat and mouse between games trying to introduce DRM and this being instantly defeated each time because the DRM code was being removed by gamers/hackers... a company started making patches, and at the end of the patch was a bunch of what looked like garbage code, binpacked to create a certain size patch. It look erroneous. But patch arrived, then weeks and months later another patch, and so it went all the while the cat and mouse continued. Then finally DRM was enabled, and the question was how? How was this done when it hadn't been caught before? The answer was that all those fragments of code weren't innocuous or meaningless, it wasn't binpacking at all... the DRM code had been split into many fragments, shipped disguised as binpacking over a lot of patches, and by the time this was figured out the whole package had been delivered.

I'm not for copyright or DRM, but I did think that an elegant move.

DirectTV Hu card updates: https://threadreaderapp.com/thread/1054964796506939392.html

same category as this. See if you can spot what's so amazing about it. https://gamefaqs.gamespot.com/snes/588741-super-metroid/faqs...

I had to Google what was so special about this guide. Now that I know, I'm trying to find some justification to spoil it in this thread.

Clearly you didn't need to justify spoiling it anyway :)

smooth :)

I didn't Google it, but when it hit me, so did your comment!

It's like the realization arrived at the same time from two different directions. Thank you!

(One of the FAQ questions in the guide helped)

If it took you a lot of time to notice it like I did: <spoiler> it's the paragraph justification

Even with this spoiler I didn’t understand it, but then I read https://twitter.com/mikko/status/1383390503635324928?lang=en and then I understood.

What's the point of a <spoiler> tag when it's impossible to not read the spoiler at the same time.

Rot13 it or something at least (unicode upside-down, etc).

op actually edited the comment like 4 time to try and make the spoiler tag work. So op, for what it's worth, ROT13 (google) is a good idea for spoilers in the future :)

Aw, the other spoilers here were at least fun.

Wow. First I thought it was only the disclaimer, but it's all the way down. How much work must that have been!?

All the way down, but sadly there appears to be an off-by-one line in the Credits and Thanks section.

As a bonus challenge: fix the spelling of "missile" (mis-spelled as "missle" throughout, in 200+ occurrences), while maintaining the layout.

And it predates any kind of AI

> But in that list of files, is a line item for the manifest file itself and along with its own hash! Something that on the surface looks completely innocuous but becomes profoundly impressive as you ponder it.

Agree this would be cool. But there would be one last realisation after contemplating it for just a little longer. The sheer number CPU cycles that would have to have been committed to achieving it would be awful. Like Bitcoin mining but orders of magnitude even more wasteful.

Or they "just" broke the hash algorithm.

Trying to find that hash by iterating on the manifest hash (as opposed to e.g. manipulating whitespace etc) would eventually succeed approximately 1/e of the time. Better get cracking!

If you define HASH(data) = SHA3(data) xor SHA3(manifest), then the manifest will always have a zero hash. This trick can be useful if you need to create a cycle in a hash-addressed list (e.g., an object-oriented class hierarchy where inheritance is by hash of the class definition—the definition of the base class would have a zero hash and could inherit from itself).

To me this can be done with a simple collision attack (assuming you can fiddle with some bytes inside the manifet file while freezing everything else in advance), which can be found under a second for MD5 with a laptop, and a few hundred thousands dollars of cloud resources for SHA1.

No, that takes a preimage attack, which doesn't exist (yet? I'm not too optimistic, personally).

It could maaaaaaaybe be done using multiple collisions that exploit the structure of a DEFLATE-compressed stream, so that you can control the extracted zip contents on a byte-by-byte basis - but I haven't figured that out just yet. Watch this space!

I don’t understand how this is a pre-image attack, as the manifest file only references itself (and not the zip file) and you can fiddle with the manifest file to your liking. To me this is the same theoretical problem as this self-referencing PNG file.

How do you fiddle with the manifest without changing the MD5?

You do want to change the MD5 of the manifest. This is what makes it a collision attack instead of a preimage attack.

Change it to what? A specific pre-determined value? That's a preimage.

This is indeed a preimage attack if the manifest content (besides its own self-referenced hash) is fixed. However this is not the case in practice: to pull off this trick you could just append some random bytes at the end of the manifest, disguised as ASCII art or something like that. The manifest would still be human readable and correct, but this would become a collision attack.

Again, to me this is the exact same problem as this self-referential PNG file, which is a very cool trick but which can be (demonstrably) computed with limited compute resources.

Appending a suffix to try to meet a specific hash value is equivalent to preimage (and is not currently possible)

One last comment though: I didn’t realize you were the author of the post (great work!!). This let me think you know your stuff, and you know something that I don’t and I need to think of all of that more carefully. So it is very probable you are right and I am wrong. Thanks for the discussion!

Again, the idea is not to find a specific hash value, but any $hash for which the property md5($manifest_content, $hash, $random_bytes) = $hash is true. You don’t need to match a specific hash value.

And you never answered how this manifest is somehow different than the self-referential png.

It seems we do not understand each other (unfortunately HN comments are not the best avenue for deep discussions) so this will be my last post on this thread as we both have better things to do than talking past each other.

Thank you for the bonus! This is the best kind of humor :)

Reply on news.ycombinator.com