Comment by maxxxxx | Hacker News Reader

Comment by maxxxxx | Hacker News Reader

After spending time extracting data and text from PDF I would also say it’s the worst file format. We have perfectly fine structured documents, convert them to PDF to lose half of the information and then we spend insane effort to get the data back somehow.

It also doesn’t render well on different resolutions.

PDF is a perfect case study how inferior solutions can become standards.

PDF isn't and was never meant as a data storage format. It is meant as a presentation format. You write a file in latex or docx then convert to PDF for sharing. This means the recipient only needs a PDF reader, not whatever tools you used to create the file. This is similar to writing a file in C and compiling to an executable. You can distribute the source, but most (outside of programmers) just want the executable so they don't have a to worry about the tooling.

That’s the problem. It was never meant as what it’s used for now. It’s bad for manuals, it’s bad for data, it’s bad for mobile devices, it’s bad for versioning. It’s only good for printing. But somehow it’s used for all of these.

Good perspective... and it made me think of a parallel with Democracy. Could the atrocity that is pdf be the "least bad" way to do all of the above in one file format?

No. It’s the worst bad :). It’s only good for printing.

Formatting matters in many cases, and PDF respects that better than any other common format.

I have policy and legal documents from 2005 in PDF/A that can be rendered identically in 2019, and likely in 2105. That isn't the case for HTML, Word or almost any non-plaintext format. If for no other reason than the US Federal Courts require use of PDF, the format will exist and be somewhat vibrant for many decades to come.

I can wholeheartedly will agree that Adobe Reader sucks, but the format solves lots of problems that are difficult to solve otherwise.

How exactly do you think your HTML files from 2005 would render differently today? Sure, your modern browser engine would probably make slightly different decisions with fonts and margins, but would that negatively impact the usability of a legal document, research paper, etc.? Meanwhile, you would have gotten things like robust search and real compatibility with mobile phones and e-readers for free.

A heavily restricted subset of HTML to replace PDF as the 'archival' format would make the world so much better of a place.

Formatting is important — all of the arguments made in favor of HTML can be made for plaintext. If you don’t care about format and want to read the document 30, 50, 100 or more years ago, you should use text.

Will HTML formatting on a 4K display will look the same as an 800x600 monitor?

Will all of the ancient display elements display the same? Will IE4 specific artifacts display?

Search works fine in PDF. Mobile is not optimal, but platforms optimized for mobile require different design considerations. Few webpages look or function identically on mobile.

How is formatting important though? Semantic formatting, sure, but again, a 20-year-old HTML file still has that preserved. The em-perfect formatting inherent in PDF seems entirely purposeless unless you for some reason need to be able to reproduce a printed copy of the document exactly. Formulas are a special case, but the great majority of documents I see distributed in PDF would be more useful as HTML, and I don't buy the argument that it would risk archivability. At the end of the day, HTML is a plain-text format which is quite human-readable even if all the world's HTML renderers were somehow lost to the ages.

The only use-case I can conceive for perfectly-reproducable layout if you are not a print publisher is in fields where it is convention to reference text by page+paragraph number; in those cases, the page number is actually semantic information so could quite trivially be encoded in the content of the document to maintain that referent.

> Search works fine in PDF.

Search works fine in PDFs that don't use hyphenation or which properly implement it, and when they don't it breaks silently in ways that are potentially disastrous.

> Mobile is not optimal, but platforms optimized for mobile require different design considerations.

And my point about mobile was you don't need to optimize for it.

>Few webpages look or function identically on mobile.

Again, it doesn't need to look identical. That is a falce requirement. There is no value in that. The only web pages that don't function on mobile are ones which have been optimized for desktop. We're not talking about general-purpose web pages here, we're talking about textual documents.

This is not some hypothetical scenario, BTW. The UK has been using HTML over PDF for public-facing documents for a couple years now, it seems to be working out for them[1].

1. https://gds.blog.gov.uk/2018/07/16/why-gov-uk-content-should...

The only web pages that don't function on mobile are ones which have been optimized for desktop.

Although it's rare to find them nowadays, this is false for pages with no CSS at all. By no CSS I mean not even the infamous proprietary viewport meta tag... which is a posteriori being made CSS. Access for instance the basically unstyled page [1] with a $1000+ smartphone of our day... and you're likely to find unreadably small font.

Now, you could argue [1] functions on mobile, but let's agree it's stretching the meaning of that term. But that's not the main point: HTML elements come and go (for instance MENU; find others in [2]), so it's clear that archival reliability is not a big priority.

We all agree that ideally the source should always be made available (mandated if tax payers' money is involved if you ask me), but that doesn't invalidate the value of a universal presentational format.

[1] http://www.qrg.northwestern.edu/papers/files/simhobby-local....

[2] https://meiert.com/en/indices/html-elements/

It’s good for manuals, which often have complex formatting and layout that necessitates precise control. It’s not appreciably worse for mobile devices than the incredibly bloated HTML that you see nowadays. It’s completely orthogonal to versioning, and PDF diff tools are quite competent. It’s not only spectacular for printing, but completely unmatched for paper <-> digital round trips.

> outside programmers

This is not true. Ask a Lawyer, Accountant or a Journalist wading through govt pdfs.

The reasons cases and investigations take forever is the time spent manually reassembling related data residing in different orgs. The same data across all orgs will show up as 25 differently structured tables. This is changing slowly but putting public tabular data in pdfs has probably cost the economy billions.

“This is changing slowly but the putting public tabular data in pdfs has probably cost the economy billions.”

Or you could says it has created a ton of jobs :)

True, PDFs should probably be thought more of as "image" files than editable documents. As a binary format, you need to use visual tools in order to view them. I have a little experience with this having built a tool to diff them: https://www.parepdf.com.

PDF is based on Postscript, it designed for print perfect output, its essentially a bunch of rendering commands for a printer its not supposed to be a word processing format that can be edited later its the final output from a word processor or whatever.

Right, but somehow it gets used for all sorts of things that are never printed in offices everywhere.

It gets printed to the screen for reading and still retains the text for searching. The ability to easily edit can actually be liability in many situations.

Its literally a replacement for paper documents, except you can search and view them on a computer and still hit print and get a great paper reproduction.

Word processing/editing has so much more going on, PDF actually removes features from postscript for security purposes, nobody wants to be sending word docs around just for viewing.

Recovering coherent text strings from PDF is non-trivial. Paragraphs are merely implicit and hyphenation must be removed, among many other problems. Anything unusual like sub/superscripts or inline math will potentially end up as gibberish or missing.

I have spent weeks on an algorithm that would recreate words and paragraphs from letter rectangles and the space in between. Fun, fun, fun. Tables are even better. Page headers and footers then complete the insanity nicely.

That has a lot to do with the program that created it, HTML can be bad to search to I mean try and index a react page.

It a better situation than OCRing a paper doc though.

The alternative doesn’t have to be a word doc; I’d prefer, open office xml docs, google docs, markdown, and just plain old rich text half the time. The other half of use cases, graphically rich report-like things, HTML or a power point would have been better. I don’t want my content to be constrained to a paper format to be zoomed and scrolled around on a not-paper-sized screen with no support for reflows.

Agreed. PDF is intended to accurately render a specific presentation format - often desktop. There is nothing quite like the melange of frustration you get when you are trying to fix something and realizing you have to open up a PDF product manual on your phone. Dragging and zooming all over the place like it's Google fucking Maps. Bonus points when the text isn't even searchable.

It's important to differentiate between the file format and software that renders that file format. Admittedly there are better and worse PDF viewers, but that shouldn't be the final (or even most important) determiner of how important it is.

Yes, I often use SumatraPDF, but there's always those super-complicated Turing-complete documents which use web-based-aunthenticated-signed God-knows-what, that won't open in anything other than Acrobat.

PDF/A-1a is a form of PDF where the text is embedded in structured way. It is a 'tagged PDF'. This is meant to make PDF files accessible. It makes it possible to extract the text well.

Some governments, e.g. Dutch government, mandate that PDFs be tagged PDFs.

I think .doc(x) is clearly the worst file format - they're not as portable as people seem to think they are, and they're not long-lasting. PDF, for all of its failing, doesn't suffer from these issues.

A better solution, perhaps, for preserving structure, would be TeX, markdown or Org files, but PDF has the advantage of not having to be compiled and being ready for presentation/consumption (possessing platform invariance).

I loved it when pdf started gaining traction.

back then there was only 1 format in the world .doc

PDF became standard because it was a SUPERIOR solution for the use cases that most people wanted.

I have to agree. PDF is the biggest pain in the rear I've had to deal with in my development career. I am also not crazy about the fact that such an important standard for storing information is so heavily influenced by one company (though PDF is an open standard at least).

I can't really say it's an open standard either since proprietary binary data can be embedded that may not be usable by the 'standard' pdf viewer.

Yes, I have had to deal with it on a development slant as well. The open source tools are rare.

Paper records are the reason systems went to databases to store information.

PDF is NOT the swiss army knife of IT-data and never will be.

PDF, What is it FOR?

Here's computerphile video that explains it

https://www.youtube.com/watch?v=48tFB_sjHgY

I'm not disagreeing with you but that does not take away from its importance. There are lots of crappy important things :)

Can’t argue about that!

"Most important" and "best" are descriptors that rarely apply to the same product. The most important beer in America is probably Bud Light.

Reply on news.ycombinator.com