OpenPDF 1.3.0 | Hacker News Reader

OpenPDF 1.3.0

github.com

191 pointsroschdal61 comments

I'm one of the maintainers of OpenPDF. Please give it a try and post your feedback. Also, if you are using iText 1.2.x, it's time to update to OpenPDF, which should be compatible.

https://github.com/librepdf/openpdf

Is there a version of PDFtk based on OpenPDF? PDFtk's original GCJ-compiled iText isn't well supported any more.

Thanks for the suggestion. I have forwarded this request to Sid in PDF Labs.

Great work, thank you for the effort you have put into this.

Is it possible to digitally sign a PDF with OpenPDF? If yes is there a tutorial somewhere or could you give a hint on how it's done or where to look at the Java API? Thanks in advance!

Here's a PDF signing example: https://github.com/LibrePDF/OpenPDF/wiki/Signing

Thank you very much!

Pdf so badly needs a standard open source reference implementation. the problem with java is that it isn’t easy to reuse in other programming environment...

something in C, or anything that compiles through LLVM would probably be a better choice i think.

Google's PDFium which is used in Android and ChromeOS is fast becoming the reference implementation to target: https://opensource.google.com/projects/pdfium

Is there a standalone client based on pdfium ?

pdfium has literally no documentation. but the good thing is that it is c which can be called from many languages.

"No documentation" seems a bit harsh - there's a Getting Started guide in docs/ [0] and fairly good commenting on the interfaces in public/ [1]

[0] https://pdfium.googlesource.com/pdfium/+/refs/heads/master/d...

[1] https://pdfium.googlesource.com/pdfium/+/refs/heads/master/p...

What's wrong with pdfium?

It uses custom google build systems that are difficult to use independent of a chromium build. Last time I tried I was unable to get it to build.

It's not packaged as far as I can tell separately from chromium and chromium doesn't ship it as a reusable library. Not sure if it has a stable interface, probably not.

It doesn't have any frontend other than chromium.

> Not sure if it has a stable interface

Their docs say

"The public/ directory contains header files for the APIs available for use by embedders of PDFium. We endeavor to keep these as stable as possible."

The rest are valid points though!

> We endeavor to keep these as stable as possible.

That sounds more like best-effort.

That is actually what I thought this OpenPDF thing was.

I feel that the name is very unfortunately chosen - it's too generic, should have had a "J" in it at least to denote that it is a Java library. JOpenPDF, perhaps.

There's already plenty of open source implementations of PDF parsing: Ghostscript, MuPDF, XPDF, Poppler, Pdfium, just to name the first few that come to mind.

All do not support typewriter or forms. This makes easier to fill up forms

MuPDF supposedly supports forms in the latest version.

I’ve never seen it work though.

Alright, so I noticed this before and just tried it again. While I can fill in forms, does anyone know how to save them? The man page says "S", but that doesn't work and I remember this causing issues for me in the past.

If you press "a" to open the annotation side panel, there will be a button to save the PDF file if the document has been modified.

OMG, MuPDF has annotations now!?

Evince can handle forms most of the time. That said, Acrobat allows things like Livecycle (XFA) forms, which are nonstandard and will break just about everything outside of Acrobat. Outside of that, pdftk can handle forms using the forms data format (FDF):

https://www.iso.org/obp/ui/#iso:std:iso:19444:-1:ed-1:v1:en

Though, it was mostly an exercise in frustration. That said, IRS forms used to work great with this.

However, long story short, Evince works most of the time.

I'm not sure whether it's just Poppler or the software that uses it (Okular by KDE), but I know that typewriter is supported (I just filled a few bureaucratic work-related form). Forms as well.

And most of them are either (A)GPL or so prohibitively expensive for the commercial version that you can't really use them in small commercial projects at all.

Sounds good to me.

the problem with java is that it isn’t easy to reuse in other programming environment.

Isn't one of the advantages of Java exactly being multi-platform?

More that Java is a platform.

You target the JVM. So Java libraries are awkward to call from e.g. C++ or Python.

I doubt many people will want to add a jvm dependency just to support a PDF library.

You can compile java to LLVM or create dll/so files.

* https://www.graalvm.org/

* https://www.graalvm.org/docs/reference-manual/aot-compilatio...

You only need to pass `--shared`. (You can also define multiple exports via public static ... and some annotations https://github.com/oracle/graal/blob/master/substratevm/READ...) of course it's a little bit akward to first create a "JVM" library and then call that library from native code, but it's not impossible.

That's a bad idea and Windows is still experimental.

I really feel this when working in python. It feels like there are a million tools that each support 1/3 of the standard.

E.g. I recently needed to convert raw OCR files to PDF -- place TIFs on each page without recompressing, place invisible text on top, add bookmarks, save. I ended up stringing together three python libraries (img2pdf for efficient image combining, reportlab to place text, PyPDF2 to merge the two), which was both clunky and slow. Second pass was scripting command line tools (tiffcp + tiff2pdf + reportlab + pdftk), which was still clunky but at least faster.

It would be awesome if there was one solid, fast implementation that lots of language ecosystems could wrap and contribute back to. I haven't looked around much outside of Python -- maybe that's OpenPDF with Graal, or PDFium, or something else mentioned in the sibling comments? But there are so many usecases for PDF, I wouldn't be surprised to find that those also fail to cover the whole field.

Y'know what I can imagine working is a package ecosystem that uses a common in-memory data structure, so you have one base package that does the reading and writing and low-level manipulation, and someone else can come along and write the equivalent of img2pdf or reportlab as separate packages that compose without any efficiency loss. That could work in any language but it feels like the kind of thing Rust is doing really well these days. It might just take some good branding around "hey, here's a low-level FooPDF data structure that is easy to write high-level libraries against -- make sure to mention in your docs that you're compatible with FooPDF." Sort of an Apache Arrow approach.

> Pdf so badly needs a standard open source reference implementation

Similarly, I would like to see more attention given to PDF/A and high quality reference implementations and libraries targeting it. It solves many of the most common complaints here on HN about PDF: no audio / video, no javascript, no XML forms, all fonts have to be embedded, etc. - and it's an ISO standard.

We really need solid support for a document format that has readers that aren't constantly getting CVE'd from supporting too much of a bloated specification.

absolutely. I developped a pdf parser myself and only discovered pdf/a on HN after almost a year working on it.

Pdf/a needs a LOT more publicity.

It would be nice to have something with a permissive license. There are a lot of commercial offerings but the costs make them prohibitive for my projects.

Go has a number of PDF libraries including GoFPDF which is actually really good and very well rounded. These days you can easily build a "C" interface to a shared library implemented in Go.

https://github.com/jung-kurt/gofpdf

My particular needs for PDF creation are somewhat unusual, so I'm not surprised that Acrobat Pro seems to be the only thing that suits my needs. If anyone knows of a PDF creation library that has good support for the various image compression schemes supported by PDF, I'd love to know about it.

Currently I'm using a 10 year old paid version of Acrobat Pro because I'm unwilling to spend another $500 to get a modern copy simply to compress images. I use it for scanning and archiving old computer manuals. In many cases jbig on 1bpp images is good enough; other times I want to use one of the more sophisticated hybrid schemes so I can have B&W or color images interspersed on the page with 1bpp text. Acrobat does a pretty good job of auto-deskewing images and doing analysis to figure out which compression type to use in different sections of the same page.

This doesn't completely fix your issue, but since you mentioned deskewing, I clean up my scanned documents using ScanTailor Advanced:

https://github.com/4lex4/scantailor-advanced

I find the autodeskewing algorithm to work well, but it allows hand adjustment as well, which I like. As I've gotten better as using it, I've been able to get the size of my scanned documents down considerably by cleaning up the scans. This includes some old manuals.

As far as the pdf encoding itself, I use both mutool, from mupdf, and qpdf. I just checked and it looks like while they both compress their streams, it may not have the same flexibility with Acrobat Pro. For me, I'll decompress, edit, and recompress streams on the files and that's been fine for my use.

That said, if someone knows of a better tool for compressing streams in a PDF, I'd be interested to hear about it as well.

I'd be interested if anything open source exists that can do jbig/jpeg2k pdfs (separation + compression) like DjVU. I think archive.org ships PDFs like this, but don't know what they're using to do it (possibly something commercial?).

It sounds like you are using Windows, but if you happen to be using Linux [1]:

    ps2pdf LARGE.pdf SMALL.pdf

I believe it compresses all of the images independently, instead of converting the entire thing to an image as I believe Imagick does (I could be wrong though). This tool doesn't appear to have any intelligent auto-deskewing though. (On a slightly different note, it's questionable whether you want to do this for archiving as it'll undoubtedly be a lossy operation.)

The reason I use this: Many websites for conferences have upload limits at around 10MB, a size you can easily reach with a handful of images from a modern device in your paper.

[1] https://www.shellhacks.com/linux-compress-pdf-reduce-pdf-siz...

I think this won't be useful for scanned documents, as the entire page will be one image.

ps2pdf does work on Windows though (or maybe it's pstopdf?), and the Linux version works directly in WSL.

I believe `ps2pdf` is just a front end script for Ghostscript. Even for `ps2pdf` there are a bunch of options settable, most notably are the DPI settings [1]. The point is that I believe it works for images and/or scalar graphics.

As the PS format was designed for printing [2], my guess is that it selects a DPI value suitable for printing. I can't seem to find the source code to back this up though. (Ghostscript has the simple web front-end that doesn't lend itself very well for viewing code in browser [3].)

[1] https://www.biu.ac.il/os_site/documentation/gs/Ps2pdf.htm

[2] https://en.wikipedia.org/wiki/PostScript

[3] http://git.ghostscript.com/?p=ghostpdl.git;a=summary

It will work for images embedded in PostScript, and resample them to different DPI if you ask (I think probably more for on-screen viewing and file size reasons than printing). I would be kind of surprised if it does anything as sophisticated as "B&W or color images interspersed on the page with 1bpp text" though (i.e. analysing full page images, locating and separating text and figure regions, and choosing different encodings for each). DjVu does do some things along those lines, but it's not widely supported like PDF.

Use hugin for deskewing?

http://hugin.sourceforge.net/tutorials/perspective/en.shtml

Look into what OpenLibrary is using. I don't remember it off the top of my head, but there is a special image format for efficiently compressing scanned pages of text. Their files are vastly smaller than jpeg at the same quality.

I downloaded one PDF and looked at its properties. It says it was created with LuraDocument PDF v2.28. Googling, it is a product of LuraTech Imaging GmbH. Following that search leads to a link that redirected to Foxit.

They have an enterprise PDF compression toolkit which clearly supports all the advanced compression schemes of PDF, but it is too rich for me: prices are not given, just a button to ask for a quote. There is a personal PDF tool for $129 but they don't make clear if it also supports their top quality PDF compression.

Try BFO pdf . It's paid but they are agood company. I also I love that BFO stands for "Big Faceless Organization" and the CEO answered support emails until like last year.

Starting at $850. Ouch; I'd just buy Acrobat Pro then, because the latter is cheaper. My needs are just for a hobby project (scanning and posting old computer manuals).

New Pure JavasScript low-level implementation to create and edit PDF files: https://github.com/Hopding/pdf-lib

Tangential question: Do you know of a method to render PDFs to HTML canvas other than PDF.js?

PDF.js seems too slow for my purposes, though I haven't tested very thoroughly yet. I'm hoping with wasm Skia now available, there might be some other options coming.

how does Skia and pdf work together? I know there is a pdf backend but the way I understand it is that it just outputs pdf instead of an image

I recently started playing around with wasm Skia/CanvasKit but I have a hard time wrapping my head around what exactly it is supposed to be used for, as well as Skia in general.

For example, Skia allows text rendering and I could naively assume that if I want to build some high performance 2D UI that Rendering text with Skia would work reasonably well.

But at what level would text interactivity like selecting text happen?

I’m afraid this is a very generic question.

Are you familiar with any pdf or text related project that uses Skia and could help me better understand what role Skia plays?

PDFium has a Skia backend, where Skia is used to do the actual drawing. Interactivity happens at a higher level.

There's pdf2htmlex, albeit it's an archived project.

We use this with great success in production: https://pdfbox.apache.org/

I tried to find a good comparison to PDFBox but came up short!

From my experience OpenPDF is better at formatting and layout. PDFBox is a lot lower level.

OpenPDF [1]:

  document.add(new Paragraph("Hello World"));

PDFBox [2]:

  contentStream.beginText();
  contentStream.setFont( font, 12 );
  contentStream.moveTextPositionByAmount( 100, 700 );
  contentStream.drawString( "Hello World" );
  contentStream.endText();

[1]: https://github.com/LibrePDF/OpenPDF/blob/master/pdf-toolbox/...

[2]: https://pdfbox.apache.org/1.8/cookbook/documentcreation.html

I've been heads down in PDF for about 6 months and looking at all the various and supported libraries.

Within Polar (https://getpolarized.io) we support pdf.js which is rather nice but ONLY supports viewing of the PDF and text extraction.

You can't create NEW PDFs.

Our plan is to use something like OpenPDF to get the best of both worlds. Editing PDFs for doing things like exporting could be done on the server and the rest done on the client.

Is this a good library for doing automated PDF form filling (like via FDF?)

If not, is there any that are recommended? PDFTK is really showing its age in an application we have (its slow, and often unreliable in large batch jobs)

Can this render PDF form fills? Because PDF.js can't read them, and I must use either PDFmaster or adobe.

Other question: Is it possible to build a document format similar to PDF around SVG?

Great, Thank you

Reply on news.ycombinator.com