As anyone able to break down why those requirements are desirable?
> MUST be the very first N bytes in the file
For every system to be able to parse it without loading the entire file
> MUST be at least four bytes long, eight is better
To reduce risk of two different binary files on the same system having the same magic number
> MUST include at least one byte with the high bit set
To avoid wrongful identification as an ASCII file (ASCII doesn't use the high bit)
> MUST include a byte sequence that is invalid UTF-8
To avoid wrongful identification as UTF-8 text file
> SHOULD include a zero byte
To avoid wrongful identification as ANY text file
> For every system to be able to parse it without loading the entire file
It also solves the ambiguity problem, zip files have the magic numbers at the end, and most other files like pdf have the magic numbers at the beginning, so you can have a file that is both a pdf and a zip file.
As a more concrete example of how file type confusion can bite you, you can imagine a hypothetical photo sharing service that lets users upload both individual images and zip files containing images; The basic structure of the server looks something like
function user_upload_hook(file):
if(is_zipfile(file)):
extract(file, tempdir)
else:
move(file, tempdir/file)
for image in tempdir:
create_thumbnail(image)
...
A malicious user has concatenated a JPEG of a cute kitten with a zip bomb. `file` reports that the uploaded file is a JPEG, so it's fed through unchecked to the server. The application's `is_zipfile()` correctly identifies that the file is a valid zip file, so the application extracts it and DOSes the server. The two different layers of the stack disagreeing on how to classify the offending file directly lead to an exploitable vulnerability.
0xDC 0xDF X X x x (0x01 0x00 | 0x00 0x01)
0xDC 0xDF are bytes with the high bit set. Together with the next two bytes, they form a four-byte sequence that cannot appear in any valid ASCII, UTF-8, Corrected UTF-8, or UTF-16 (regardless of endianness) text document. This is not a perfectly bulletproof declaration that the file does not contain text, but it should be strong enough except maybe for formats like PDF that can't decide if they're structured text or binary.
X X x x: Four ASCII alphanumeric characters naming your file format. Make them clearly related to your recommended file name extension. I'm giving you four characters because we're running out of three-letter acronyms. If you don't need four characters, pad at the end with 0x1A (aka ^Z).
The first two of these (the uppercase Xes) must not have their high bits set, lest the "this is not text" declaration be weakened. For the other two (lowercase xes), use of ASCII alphanumerics is just a strong recommendation.
0x01 0x00 or 0x00 0x01: This is to be understood as a 16-bit unsigned integer in your choice of little- or big-endian order. It serves three functions. In descending order of importance:
It includes a zero byte, reinforcing the declaration that this is not a text file.
It demonstrates which byte ordering will be used throughout the file. It does not matter which order you choose, but you need to consciously choose either big- or little-endian and then use that byte order consistently throughout the file. Yes, I have seen cases where people didn't do that.
It's an escape hatch. If one day you discover that you need to alter the structure of the rest of the file in a totally incompatible way, and yet it is still meaningfully the same format, so you don't want to change the name characters, you can change the 0x01 to 0x02. We both hope that day will never come, but we both know it might.