Comment by vagab0nd | Hacker News Reader

Comment by vagab0nd | Hacker News Reader

Just recently I was tasked to convert some huge html pages (with lots of small entries) into a pdf file. The requirements are "fully automated solution" and "pdf must look the same as the page when viewed in a browser". Probably takes less than five minutes, right? I thought the same.

Wrong.

Chrome/Chromium crashes due to hard coded memory limit in V8.

Firefox has no command line option for printing pdfs.

No other libraries render the pdfs correctly because they are not full-fledged web engines.

So what were my options?

1. Read/understand Chromium source, recompile to lift the memory limit.

2. Read/understand Firefox source, recompile to add a command line option.

3. Use some UI testing framework to automate pdf printing in Firefox.

Eventually I did 4, which is split the html files into smaller chunks, convert and re-combine. Of course the problem is how do you know where to split the html so that it's at the boundary of the page? The solution is to do a binary search for the number of entries to put into each chunk when the number of generated pdf pages changes. What a pain.

In 2014 we used wkhtmltopdf[0] to generate PDF copies of Cloudfoundry docs for every version every release, and maybe that's what I'd reach for now. Not sure if Qt WebKit has similar limits as Chromium.

Not that you asked, but I am sitting here silently judging whoever let those pages get that large. Enough html to cap out RAM? Chesterton's Fence dictates that I presume your upstream's hands were tied, but wowee!

0. https://wkhtmltopdf.org/

Thanks for the suggestion. Yes I've tried wkhtmltopdf. It works, but unfortunately it doesn't interpret the CSS correctly. So the end result looks very different from the actual webpage.

Just because Firefox doesn't have a command line option to print to pdf doesn't mean it's not automatable; you could (have) automate(d) Firefox’s UI instead

Have you considered Playwright? It offers very straightforward APIs to do exactly this.

This is the way. Firefox may not have a CLI arg for saving PDFs, but they expose it through DevTools API and it can be automated with Playwright, Selenium (or plain http requests) etc

I use PrinceXML for converting long HTML into PDFs and haven’t had trouble with large documents, though I don’t know if we’re in the same ballpark in terms of file size or element count. It’s expensive but is a one-time purchase (and I think it’s free to use personally and to evaluate). You can also use it indirectly through DocRaptor (basically a PrinceXML SaaS with an API), though I’ve never tried it.

The overall size is not that big. The problem is that the html contains lots of small <div>s. But yeah I didn't bother trying any paid services. I probably should have.

Run the browser’s native print to pdf and save the result.

This destroys the CSS, forces paper sizes, etc.

For anybody else having the same problem: Orion Browser for MacOS is AFAIK the only application on any platform that is able to save HTML pages as PDF with perfect results. It is based on WebKit and not automated, but maybe it can be scripted.

Pandoc generally works just fine: https://pandoc.org/

Reply on news.ycombinator.com