Hacker News Reader

Comment by mettamage | Hacker News Reader

What's your favorite PDF feature that causes a brain meltdown?

I've read a few comments on HN how PDF is, well, not developer-friendly. If people are interested in providing some more examples here, I'd be curious to know!

View on HN

In the early 2000 I coded a PDF library for an industrial printer suite. (Print, proof, impositions)

I personally think the structural PDF format is a really great format. It's entirely ASCII-based, a pure text format, yet it can embed arbitrary binary data and compress that data. The actual structure is simple and support just enough functionality, like a tree of object, dictionaries and arrays, unicode strings, date formats, etc.

I think if you limit yourself to pure structural PDF woulde have been a great format to standardize upon, much better than JSON or XML. It';s richer than JSON, simpler and saner than XML. Again, it's top-notch ability to embed binary is great. It has other great characteristics, for example you can update anything just by appending.

The ugly bits are in the "semantic" PDF: the page descriptions, media, etc. Even then, the early version of PDF were nice, mainly just simplified Postscript.

I'm of a similar opinion but I'll say the format is quite good, but the many and varied implementations are often not.

A common case is clients who use utilities that generate single customer documents then merge them into a bigger file for bulk print and mail (bills and statements, not identical copy). Without fail that results in thousands of similar but different subset fonts whereupon most printers I've encountered eventually fail due to memory issues.

Typically this leads to a discussion about "I can open it on my computer fine" and bending over backwards to find a workaround. Merging and consolidating these fonts doesn't seem to be a simple task, although some tools claim to work some of the time.

Something that scopes object resources for disposal could be nice in the PDF spec (maybe it exists), but something like a LRU caching mechanism on the printer would potentially resolve this too.

Being able to do SQL queries to remote servers, upload form contents directly to a server, embedded 3D models and being able to have a fully featured page embedded Tetris game due to support for JS.

Having said that (and worked on a commercial PDF library), despite all the cruft that came with age, it's a well built format that survived the test of time with good reasons.

Can chat with you about PDFs? devon at digitalsanctuary dot com

Nice

I worked on a Pdf with inbuilt tracking solution that updated the form layout using ActionScript based on the workflow status and the role of the user (ie the line manager had a different group of fields in the form to complete like their signature while viewing what the initial requestor had entered) Lots of callbacks to the server saving in progress data and updating the status of who had the form, who was next based on department and emailing it to that person if it passed validation.

An initial fun discovery was that you could force the form to download and replace itself with the latest version even if they had just opened some old file they had on their pc.

Acrobat can read fantastically corrupt PDF files none of which are covered by the spec. The endless surprises induce a special kind of madness.

Streams just suddenly end? That’s ok. Totally corrupt xref tables? Ok. Incorrect image headers? Ok. Unrecognisably mangled Type1 font formats? Fine!

That's great because it creates client expectations regarding what my PDF application should support. Implementing the spec is not good enough, you have to do what PDFium or Adobe do.

On the other hand, if they never supported those broken PDFs to begin with, we wouldn't have them in the wild and wouldn't have to deal with them.

20 years ago I were more surprised when I got a PDF with correct xref table than with broken one.

I think one of the biggest pain points that developers hit that hasn't been mentioned here is content extraction.

A lot of the time developers want access to the text inside PDFs. Unlike HTML or formats like MS Word (XML or old binary format) getting "text" isn't really possible.

Most "document" formats have the concept of words or strings: a set of characters separated by whitespace. PDF isn't a "document" format in that sense - it's a page description language. Instead of strings of text you have character glyphs positioned at a particular location.

If you want to "read" the text, you have to work out the orientation (which can change throughout the page - think of table header alignment), and use some kind of heuristic to guess the word spacing based on the font and character spacing.

There's also this whole thing with clipping, where some text can be hidden behind other objects (or off page) so you have to try to deal with that.

There's lots of libraries that try to do this for you, but there are lots because none get it right 100% of the time...

- Remapping font tables to different characters for to reduce code usage.

- Clipping path logic, you can write text outside of it, which makes it effectively invisible yet it will show up if you try to extract the text.

- Anything regarding the graphicstate stack, it's a pain to debug.

- Extracting content from AcroForm/JS "XFA" forms

PDF is great format for printing, it's just a pain for pretty much everything else.

This is also my list. Except for the forms, that's one I don't have to deal with.

My other one is the use of multiple subset fonts that are actually the same font with a different subset of glyphs that you want to merge back together.

"Identical" PDFs are not necessarily byte-identical; you can't just check equivalence through checksums. (AFAIK, it's been a while, feel free to correct) I don't remember a good way to normalize/disambiguate, at least I never attacked the problem long enough to have learned.

EDIT: oh yeah, I'm pretty sure it contains Mail, just like Zawinski said

At least it's probably better than MS-Word's internal format ... (?)

Microsoft Word stores XML documents inside a zip archive. There is a detailed specification of the format available: https://docs.microsoft.com/en-us/openspecs/office_standards/...

I think he was talking about the classic .doc format which was a clusterfuck and not the open XML.

If I remember correctly the XML format was just an XML-encoded version of the binary counterpart. Including all or most of the bugs and weird hacks.

with the previous format being essentially a memory dump, i'd say that's progress

That’s correct - I worked with MS team that documented old formats, and they said that sometimes they don’t have people left who knew what specific struct was intended for - although that was mostly for people PowerPoint and Visio, excel and word was better documented

Actually Excel was the only one that had official, freely available documentation for the old (now legacy) file format.

You don't remember correctly. Word's docx format is far more intelligent than openoffice ODT, despite propaganda to the contrary. With one exception: word's zip files don't have a convenient magic header. The way it works with ODT, and a bunch of other formats is that you put an uncompressed identifier file (`mimetype`) as the first entry inside your zipfile. At byte 30 (of your zipfile) you then get `mimetype$THE_MIMEMETYPE`. This is a nice trick and works for any zip-based format. Sadly, docx does not do that so you have to go by file extension or look at (more of) the contents of the zipfile.

IIRC the original doc (and xls) formats were unwieldy mainly because of performance requirements. In order to save and load fast they were basically a bunch of binary dumped structs.

I'm not saying it's beautiful, but isn't Ms-Word's internal format basically a series of XML files that are zipped up?

The old .Doc and .xls files were a bad format, but my understanding is that since Office 2007 the format is generally much better.

To be fair the old doc format was conceived in DOS era and memory efficiency was a primer back in the days.

and if they hadn't waited so long to update to a sane file format no one would complain but they waited until 2007 to fix the format long after the dos era memory excuse had long ceased to be an issue. even then they only did it to allow them to shoe horn their format in as the iso standard after one was already selected bribing there way through the process.

Ms office files prior to office 2007 were mostly memory dumps of specific components, wrapped into composite files aka OLE2 storage - their content varied depending on office versions and often locale

Its a wonder alternative editors ever supported the format..

Reply on news.ycombinator.com