Hacker News Reader

Comment by alephnerd | Hacker News Reader

This is a great UX addition. Why did it take them so long?

How would you do it quickly?

For example, HTML isn't divided into numbereres pages while PDFs are. A lot of latex interacts with page boundaries. Figures tend towards the tops of pages. And there's \clearpage. And the reference list might say which page each citation appeared on. All that stuff needs someone to decide how to handle it and then to implement that handling. Like... what value does \pageheight return? Sometimes I resize things to fit the page height, and if it was doubled then I should have resized to fit the width instead.

Latex is a very complicated programming language for creating documents. It is not easy to create a new backend for it.

As a glimpse into the very tip of the iceberg, this diagram is https://tex.stackexchange.com/a/158740/ generated with 100% Latex code.

The conversion is still very error-prone. It can't convert a lot of packages, and the last paper I read, StarVector, half the HTML version is just missing. (I think it hit an error at a figure of some sort.) I reported an error, but I've been reporting errors against the ar5iv and abstracts for years now and the long tail of problems just seems like an incredible slog.

Where are the computer vision people? This is the perfect type of problem for multi modal LLMs

Except that the errors made by an LLM might be harder to spot then converter errors that typically are very blatant, and don't usually alter text (perhaps just drop parts of it).

Also, a bug in a converter is conceptually much easier to fix than to re-train your LLM.

I am not sure that AI in it's current state is useful when "high fidelity" is required.

Can confirm. From an ar5iv standpoint, 2.56% articles currently fail to convert entirely, and 22.9% have known errors to the converter. That leaves 74.5% of nominally usable articles. This success rate is noticeably lower for the newest batches of arXiv submissions, as the converter hasn't caught up with the most recent package innovations.

We have a plan in place to meaningfully fall back for unknown packages, but that will take at least another year to put in place, and likely another couple of years to stabilize.

Meanwhile, there is some hope that with arXiv launching the HTML Beta we will get more contributions for package support (LaTeXML is an open source project, with public domain licensing, everybody benefits).

But again the original point is spot on. Coverage will be hit-or-miss for a while longer yet, for an arbitrary arXiv submission. The good news is that authors could work towards better support for their articles, if they wanted to.

Almost universally, we prepare conference papers as LaTeX files made to export to PDFs which fit within the conferences template.

It's nontrivial to export this to HTML in all cases, and even then, nobody is asking for HTML from us even though we all want it. I'm guessing Arxiv is using some kind of converter which _usually_ but not _always_ works.

That said, this is a long time coming and PDF as the standard should've died a decade ago. I wish I had this when I was in my PhD program.

Because this is a rather conservative field with little dependency on the general public, so without much interest in hepling disseminate the knowledge broadly & accessibly (relative to other priorities, not absolute)

Reply on news.ycombinator.com