The story of the PDF (2018)

Imagine a simple file format that doesn't support text wrapping, but allows you to specify elements as (x, y, s) where (x, y) specify a position, and s is a string that will be written left-to-right, truncated at the edge of the screen.

That's a simple file format, right?

But inserting a word somewhere early in that document would change the string within every element in the rest of the paragraph. And maybe move the y position of every element later in the document.

That would be a PITA to diff. Even more so if the document has more than one column.

Well, probably Microsoft XPS, which was actually a fairly well designed format. But Microsoft didn't have the fight in them to really push it as a competitor to PDF. In part, I suspect b/c it's hard to justify investing a lot of money in your competing document standard as there is not much revenue you can derive from it. As of 2018, Microsoft no longer bundles XPS support in Windows 10.

Convert to text with: pdftotext -layout

Other than compression and such encodings, PDF files are actually text files, with a drawing model largely based on PostScript but without the programming. If you want to diff them, use `mutool clean -d -a` to first turn them into pure ASCII text.

That said, since it's a "baked" layout format, if one word pushes the rest of the text forward, everything after that will show up with changed coordinates. It's closer to a vector image format like SVG than a markup format like HTML or ODF.

There are also things like font subsetting, where removing a word that was the only use of a character, or adding a word that uses a new character, might change the font data to add/remove those characters.

I always get amused by proving those people how to edit PDFs.

It is the same logic that documents sent by Fax are legally binding but the same document sent by email not.

Sadly some information I only get as PDF's, so I have to scrape them. Easy right? It can be, if the PDF is structured sanely. But PDF isn't some well defined data structure for laying out the page, it's a Turing complete stack based computer program that can do whatever it damned well pleases. The font tables don't necessarily have ' '=32, 'A'=65, 'a'=97. Why not optimise it and get rid of all those gaps, so now ' '=0, 'A'=30? And it doesn't have to be drawn in any sane order. It can be just a mess that makes even copy & paste near impossible, and some are.

Did we really need to invent a DSL that has to be executed every time we wanted to view page? I remember it being pushed as a cool solution at the time. It doesn't look so cool now. SVG would be an improvement.

No, those are formats with completely different scopes. They don't compete and are essentially non-interchangeable.

> There was a time when papers, books, resumes, contracts, etc. almost always came as Word.

There was never such a time. I mean, sure, you could (and can) send people Word/LibreOffice documents, but things that needed some reproducibility and finality [1] were distributed or published is MS-Word format - almost ever. Postscript used to be pretty popular though.

[1] - Yes, PDFs can be edited too, I know.

Diffing PDFs at the textual level is a much harder problem though since lines of text need to be reordered and concatenated with each other. Unfortunately there is nothing built into the format that allows you to know what line belongs with what other line beyond guesswork.

• Because one was top heavy, the other was not.

• Because one was a document format shared between an application that creates ne one that displays, the other was a whole server/protocol/client stack.

• Because one would insist on rigidly paginating it’s content as output by the generating application, while the other defined content that would be streamed to your client and allow it to adapt the content to your display and reflow it’s (admittedly primitive-looking) text & cetera.

• Because one was designed for use within corporations to distribute documents, while the other was intended to allow collective authorship beyond corporate confines and that this consumer/researcher technology would later seep back into the corporate domain and possibly screw up their plans.

Another way to look at it: if the Adobe folks were angsty, irritable, annoyed, or otherwise flustered, it’s probably because they knew (some?) of the above (and perhaps more) and realised that they were going to have a fight on their hands.

It became open source in 2008 so it’s definitely here to stay.

And indeed you are correct! A time shall arise, sooner or later in the future, a moment when the last PDF file is created, as well as a moment when a PDF is consulted for the last time. Depending on your definition of format obsolescence, this might be well beyond its expiry date, or it might actually mark the moment of death.

(Let’s forget that the Apple lineage of OSes derived from Display PostScript-using NeXTstep such as OS X [latterly macOS], iOS, iPadOS, watchOS and tvOS all use PDF as a mechanism for drawing primitive sources onto the screen.)

Anyway... after that long preamble, statements like these remind me very much of Goldfinger’s famous quote, and in honour of Sean Connery’s passing yesterday I will allow myself to elucidate:

Bond: “Do you expect me to talk?” Goldfinger: “I expect you to die!”

The latter being a very reliable expectation, but one that can sometimes take a lot longer to come true than the utterer might have in mind when they make the assertion.

1: https://techcrunch.com/2020/09/23/adobes-liquid-mode-uses-ai...

There are other formats for flowable text in screens.

For one thing, PDF can encode information that rtf cannot.

There's also lots of approaches to document layout (the underlying descriptions of what should appear, not just different styles).

Pedantically, the analogy works somewhat better for postscript than rtf, but not really, except maybe the bmp->png part.