Comment by wongarsu | Hacker News Reader

Comment by wongarsu | Hacker News Reader

Parent Full thread

The article kind of glosses over the PNG signature. There is actually a lot of thought behind using these exact 8 bytes:

The first byte is 0x89, meant to tell any text editor that this isn't a text file, and to prevent text files from being recognized as PNG. It also detects if you mistakenly transferred the file in some ASCII file mode that clears the top bit.

The next three bytes are PNG in ASCII, meant for humans looking at it in a hex editor.

The next two bytes are a Windows-style line ending (CR LF, or 0D 0A). This breaks the signature if the file mistakenly undergoes Windows->Unix line conversion

The next byte is 0x1A - End of File. This causes DOS tools to stop printing the file.

The last byte is a Unix-style line break, to detect Unix->Windows line conversions

To make the validation process complete, a process can check the integrity of the last chunk (IEND). Not only does it provide a fixed sequence of 12 bytes you can use to check that the file is complete, the fact that its length field is 0 allows you to verify that zero bytes were transferred correctly. Of course you would likely notice anyways because of the chunk checksums, but IEND allows you to fail fast and makes it easier to diagnose the actual issue.

http://libpng.org/pub/png/spec/1.0/PNG-Rationale.html#R.PNG-...

I thought this was brilliant when I first learned it, but I've since come to think it's ridiculous. JPEG doesn't have any of these error-detection features, have you ever missed them? It seems to be the same mindset that gave us double-checksummed IDATs.

Perhaps there was a world once where this made sense, but we aren't living in it now.

PNG definitely came out in a world where this made sense. A FTP client was set in the wrong mode could easily corrupt everything.

Interesting, clever and beautiful. Now, why did they have to be so defensive against corruption? What purpose does it serve?

If I had to detect corruption I would probably use something like CRC, it seems more complete and more robust.

Not all modem standards had built-in error detection. If you got garbage or drop outs because of line noise or someone else trying to use the parallel phone, that was all you got. The sending side did know nothing about that. For reliable transfer of binary files, software on both sides had to implement X/Y/Zmodem or some other protocol which added metadata for correctness checks. Files that were cut too soon when copying over the modem or from one floppy to the other, or because the process moving data died for some reason, were also not uncommon.

Historical 7 bit protocols were still used. Even though systems of their origin were a thing of the past, and links had 8 bit characters, existing software handling those protocols often mangled the “spare” highest bit for its own reasons.

In simple terms, the PNG header allowed to check that the file had always been handled in binary mode, and maybe even to revert some types of the damage. Those were the last days of distinction between terminal-style “text mode” processing (with hardware or software components in the middle silently converting the code pages, line endings, and control codes if they were different on two sides), and “binary mode” direct copying, soon to become the only expected option. People writing binary data to files today almost never invent the means to escape certain “special” values in anticipation of possible transfer over this or over that.

Reply from a sibling comment of yours: https://news.ycombinator.com/item?id=39246642

> A FTP client was set in the wrong mode could easily corrupt everything.

When PNG came out that was actually a super common thing that happens. And specifically it would corrupt the newline characters.

Reply on news.ycombinator.com