Hacker News Reader

Comment by jessriedel | Hacker News Reader

I've said it before and I'll say it again: the single most cost effective way to speed up math and physics research by a philanthropist would be taking a few million dollars and hiring a team of developers to bring LaTeX into the 21st century: Take the 100 most popular packages and fold them into the main code base (so I don't need to install conflicting packages just to get three columns!). Create a non-buggy desktop IDE/wordprocessor that works seamlessly using a traditional interface or a LyX-like interface. Merge/buy/replace Overleaf with an online service that syncs reliably with the desktop IDE. And, most importantly and difficultly, create a standard way for generating beautiful web pages (including mobile) from the same LaTeX++ files that can be used to generate beautiful PDFs.

The annual NSF physics budget is roughly a quarter billion dollars. This only needs to make NSF-funded physicists 1% more productive over a decade to justify spending $20M, and the entire world would benefit.

EDIT: Lots of commenters worry that this will end up like various closed-source for-profit tools like Google Docs. But for a closer analogy they should look at some of the excellent free open-source science software being produced with philanthropic funding such as Zotero. Physicists will only adopt it if it makes their lives easier (and maybe not even then...).

EDIT 2: There are some pretty strong parallels between the commenters suggesting this can already be achieved with homegrown tools and the infamous HN Dropbox comment.

View on HN

Knuth wrote TeX as a labor of love. The surest way to kill everything good about it would be a few million dollars, and a team, and to turn it into fucking Google Docs for science.

Ask Knuth to name one person who has the clearest vision for the future of TeX, and give that person a stipend to work on it for the next ten years, or for life, and you might actually do some good.

Leave TeX alone, just build better tools on top of it.

I mean, that's what they were proposing; LaTeX is "on top of TeX", and LaTeX is what they were suggesting should be fixed.

Yes, but I was responding to a comment about TeX.

While it's possible to build a lot of things on top, maybe it'd be worth making some minor updates to the core TeX engine itself, to bring it into the world of Unicode and OpenType fonts...

Oh, there are a couple of such candidates, actually, built with very different philosophies: there's XeTeX, intended to graft Unicode and OpenType support onto TeX with as little disruption as possible, such that it "just works" with minimal effort; and there's LuaTeX, which "deconstructs" TeX to give you an infinitely-flexible typesetting toolkit that can do just about anything, if you can figure out how to wield it.

Good points.

I think the idea would be more like Jet Brain’s IDE for LateX. Ofcourse I have doubts if some team can pull this off well.

Have you tried the TeXiFy IDEA plugin? It's well-maintained and has saved me a bunch of time.

https://github.com/Hannah-Sten/TeXiFy-IDEA

https://plugins.jetbrains.com/plugin/9473-texify-idea

That's...amazing. Didn't even think to look there. Will totally be checking that out.

Can you elaborate on what you think is bad about Google Docs? Aside from the data privacy concerns from the fact that it's owned and operated by Google (which I agree with), it seems like a fantastically usable, available, and powerful product. What do you fear would happen to LaTeX in this situation?

2 things, 1) Why do you consider Google Docs to be any different than Word? They both serve the same market, so you might as well be asking why not standardize around Word. And 2) given the intended uses of Google Docs/Word, the biggest reason not to use it for research are exactly the set of asides you mentioned.

That seems irrelevant in this context as both Microsoft Word or Google Docs would fit the analogy in the same way.

> you might as well be asking why not standardize around Word.

The comment I was replying to specifically called out Google Docs as bad, so that was what I was referring to. I agree that they appear to have mostly-similar features (though, to me as someone who doesn't use either as a Power User, GD seems to be more focused on online collaboration, and MW on local editing, the output of which is then shared)

It's a fantastic product for locking the organizational knowledge of countless teams in the data centres of the biggest (publicly traded) enemy to privacy the US has ever seen. This would be a disaster for science, would suck the life out of the TeX ecosystem, and would be a perfect acquisition target for evil science publishing parasites (you know the names) to further strangle the scholarly discourse in the name of convenience. If the files don't live in my hard drive and aren't published to my web server, I don't want it and will fight against it.

And before someone asks, yes, I feel similarly about GitHub, and wasn't the least bit surprised when it sold itself to the self-declared enemy of open source of previous decades (now reformed, of course!).

Yes, these things are convenient, they have great UX nudging you toward what serves the interests of whoever owns them. I'm not arguing against the usability of Google Docs but against transplanting that model to a community built on entirely different principles.

OK - thanks for clarifying! I was wondering if there was some objection beyond privacy and control (which are certainly sufficient in-and-of themselves!).

> evil science publishing parasites (you know the names)

Not being part of the scientific community, I don't - but I trust you that they exist!

I do totally understand what you mean, but I don't think you should dismiss the idea this swiftly. This is effectively a dictatorship vs democracy argument. I tend to suspect that neither solution would provide exactly what you wanted unless you were the dictator!

If you accept that premise, the decision has to be: which system would serve the most people the best? I'm inclined to think that a team with a budget of $20M would probably get the closest.

If I had a few million dollars budget for improving latex, I'd spend 90% of it UX research. What are people using it for? What are the biggest pain points? Which easy wins are there?

Then try to modify it as little as possible while improving things.

Or just improve HTML text rendering such that Latex isn't necessary to generate beautiful documents.

And add better support for concepts like page numbers, page headers, page footers, which while theoretically supported by HTML and CSS, aren't well supported in practice.

Given that many people use Python now for scientific purposes, and it's a straightforward language, to add extensibility to this Latex replacement, maybe just use Jinja templates and Python packages.

You are greatly underestimating (1) the effort it took (in committees, over decades) to get CSS where it is now (and DSSSL, XSL-FO, and all the related work in this area) and (2) the vastness of the gap between where the Web platform is after all these years vs. where TeX already was before the Web even existed.

Obviously everything that can be built by an individual is better that way than if built by a team. But at some point the amount of work becomes too much for one person to finish in a reasonable amount of time (e.g., a lifetime, or before their project becomes out of date).

What I'm describing is not do-able by one person, and the result of refusing to coordinate a team around the project (and instead having a hodge-podge of tools built by various people) is the current LaTeX ecosystem mess.

People do dissertations, articles, etc using Word/Google Docs, etc

Of course it's a work of love, but it's a tool from the 70s

Clinging to the past might be interesting but in the end it's that, a tool

Yes, it is ostensibly a tool from the 70's but it is also the most complete, correct, and featureful typesetting engine that exists.

TeX's downfall is the macro system. It makes it too easy for people to pile on poorly written abstractions that make working with any given document a nightmare because it has its own DSL. But because those packages and macros grew organically by researchers solving problems for themselves they are too practically useful to just drop them. So there are lots of annoying unintuitive quirks that you end up just having to put up with.

Yes. And, yes, there are better end user tools, for many use cases, for creating documents to be typeset than coding them directly in TeX, but (starting with LaTeX and moving up to many more modern solutions) a lot of them are built on top of TeX (and some even on LaTeX) rather than replacing it.

If it ain't broke, don't fix it.

Considering what "modernizing" other parts of the tech stack has meant, I would expect a 21st century LaTeX to be hundreds of megabytes, require running a buggy, slow electron app, and phone home to a variety of SaaS providers when you use it.

Edit: Someone beat me to it by suggesting VSCode as an IDE.

Hundreds of megabytes would be a considerable improvement for me. I think my last installation of LaTeX and the few packages I depended on was around two gigabytes.

I think you are confusing RAM with storage.

I suppose they could have meant that, but I think it's pretty reasonable to interpret that as having meant size on disk

You’re right. The original post was not clear.

And it's already slow and needs to run multiple times to generate a table of contents and bibtex

Edit: phone autocomplete typo

Generally I use a Makefile that looks like this

```

SRC :=$(wildcard .tex .bib)

Paper.pdf: $(SRC)

pdflatex Paper

bibtex Paper.aux

pdflatex Paper

short: $(SRC)

pdflatex Paper

clean:

rm -f .glo .log .dvi .gls .toc .aux .ist .out .glg .pdf .bbl .blg .lof .brf *.sta

.PHONY: clean

```

I hate that you have to run things multiple times to get references right. Really it should just be `pdflatex Paper` and you should be done.

Every once in awhile I just get references that don't want to stick. Same with images. [ht!] and floatbarriers only go so far. As much as tikz is a pain (and beautiful), I'd definitely invest more time into making image placement and references easier for the average user.

EDIT: code has a lot of wildcards before every extension but I'm not sure how to escape those on HN

Isn't this what latexmk does for you?

I don't use things like bibtext so I don't know how it handles that, but latexmk works perfectly for normal use cases, and ensures that you never have to run it more times than you need to.

I didn't know about this before so I can't answer, but it looks like yes. Thanks!

I've enjoyed using TinyTex, which is sort of the Arch Linux of TeX distributions: https://yihui.org/tinytex/

> the single most cost effective way to speed up math and physics research by a philanthropist would be taking a few million dollars and hiring a team of developers to bring LaTeX into the 21st century

This might be a niche opinion, but I think Rmarkdown & Jupyter get us a non-negligible portion of the way there. You can use LaTeX chunks when you need equations, markdown syntax when you need formatting (IIRC this is a pain with LaTeX), and executed code chunks to (e.g.) create and insert a graph. (One advantage of Rmarkdown is that you can have as many languages in a single document as knitr can handle, each its own chunk [0]).

Neither is all the way there yet, but, as Yihui Xie puts it [1]:

> I have a dream that one day all students and researchers will forget what "formatting a paper" even means. I have a dream that one day journals and grad schools no longer have style guides. I have a dream that one day no missing $ is inserted, and \hbox is never overfull.

[0]: https://bookdown.org/yihui/rmarkdown/language-engines.html

[1] https://twitter.com/xieyihui/status/1022873179532996609

bUt MuH TypESeTtInG!

I think you're absolutely right. But people seem to value a specific flavour of aesthetics over easy to understand and reproducible research.

I guess that's maybe partly caused by the publication pressure, once you start to pump out papers for pumping out papers sake, there is little incentive to go beyond the superficial.

I have a dream that one day we'll get a language for typesetting that isn't a markup language.

Rather, this language would be designed to come after writing, in the same way that flagship Desktop Publishing software (InDesign, Publisher) make the assumption that layout happens after writing. (Sure, you can write inside the layout software, but you're not supposed to. They're assuming e.g. a journalist–editor–layout–typesetter pipeline.) You'd have a "layout file" (written in the typesetting DSL) that has separate "non-typeset rich-text" files linked to it via file/URI references (just like Desktop Publishing apps do.)

The person doing layout would never modify the linked source files; if they wanted to e.g. change certain character sequences into filligrees, that'd be something stored entirely as an annotation in the layout file, not as markup in the source file.

In that world, scientists just write the "non-typeset rich text" (for which Rmarkdown / Jupyter would be a great fit); and then it's the job of science journals to produce the "layout + typesetting annotations."

And hey, then journals would actually have a reason to exist! They would be to science as physical magazines are to journalism at this point: a place to find the articles you can already read online, but typeset much better.

Its not as straightforward. Converting a document from one column to two, often means that some equations have to be intelligently broken up across lines. There have been times when I have defined new variables, so I would not constantly have to break the two column layout for long equations.

Figure sizes are also a function of document fonts and size.

I'm not saying you shouldn't be able to modify the way the source appears, even going so far as saying "there should be a new variable here." I'm just saying that those modifications should live in the layout document, rather than in the source itself.

Think of it like an archivist restoring a painting. First, they add a (removable) clear-resin barrier layer; and then they do restoration work on top of that barrier. In essence, rather than the paint they're adding becoming part of the painting, it instead becomes part of a "wrapper" for the painting, which can always be "unwrapped" back to the source.

Except this analogy isn't perfect, because you can only override a painting point-by-point; whereas a layout document can "losslessly" cut a linked source up into chunks and put each chunk in a different place, repeat chunks, insert things between chunks, etc.

A better analogy, actually, might be a DAW project. The input audio samples are immutable; the DAW project takes samples from the inputs, applies changes to those samples, and then sequences the results.

And note that a large part of the point of this workflow (in desktop publishing, at least) is that the upstream can continue iterating on the linked document as the downstream lays it out. Those changes may break the downstream layout, but they usually don't, because the downstream layout has usually been defined by constraints.

For example, in a newspaper article, if the source-text of the article gets more wordy, then up to a point, the layout engine just packs those same paragraphs tighter on the page it already decided they appear on, while keeping the paragraphs that appear on the next page spaced as before, and not allowing any lines from the paragraphs on page A to migrate to the flow-box on page B. But at a certain point, it switches strategies, moves an entire paragraph from A to B, and then relaxes the spacing of A back down. At that point, the layout may have been broken (because now B might be either too long to fit its flow-box, or unreadably tightly-spaced.) But that happens surprisingly rarely, compared to constraint-based reflow "just working" to absorb linked-document changes. (And note that either the writer or the layout designer can aid this process by inserting explicit page/section breaks that add weights to these constraints, to keep reflow from flapping lengths around.)

-----

Note that everything I'm talking about is already how a very domain-specific kind of Desktop Publishing already works: "engraving", a.k.a. sheet-music publishing. The score itself (the thing that contains information that could be converted to MIDI) and the engraving details (the way to present the score to a human) are kept separate in most engraving-tool formats.

> Note that everything I'm talking about is already how a very domain-specific kind of Desktop Publishing already works: "engraving", a.k.a. sheet-music publishing.

You may be amused to learn that there's a TeX for sheet music -- Lilypond [1]. I don't believe it's commonly used in the industry, but it can generate decent-looking output.

[1]: https://lilypond.org/

Also check out Noteflight:

https://www.noteflight.com/

That sounds a lot like EPub, where you have html and CSS.

It's an interesting idea, but a counterpoint would be: Why all this work for a dying medium? Paper is on it's way out, and there are so many different screen formats (including E-Inc) that dynamic layout is the way to go.

> That sounds a lot like EPub, where you have html and CSS.

Eh, kinda. CSS requires the assistance of the HTML, though; HTML is "the boss" in the relationship, in that it gets to define where markup boundaries are, declare what CSS classes and identifiers apply to what elements, etc. So "I write the HTML, you lay it out with CSS" isn't really a valid pipelined workflow, if you expect to do anything fancy with your layout.

In GUI desktop-publishing programs, the layout document is "the boss." It's as if CSS could target its rules by byte-slices, rather than needing HTML to hand it ready-made selectors. You can take arbitrary bits and pieces of the linked source, style them how you please, and throw them onto parts of pages as you please.

Consider: in GUI desktop-publishing programs, a "pull quote" is not a second copy of the text in the source. It's just a second reference to the same text-slice of source-text, within the layout.

> Why all this work for a dying medium? Paper is on it's way out.

Layout/typesetting isn't just for books. Posters need layout. Billboards need layout. Business cards need layout. None of these are dying out in the least.

Heck, even for screen media, text within videos (e.g. title cards, ad copy, etc.) has static layout, and so benefits from layout/typesetting.

There will always be a place for tools that allow someone to efficiently describe (or hint) the best way to show text (on a screen, or on a page) for best readability — or best impact.

And even leaving aside the practical uses of text layout, I should note that kinetic typography is also an artform, done for its own sake—and one that often requires the assistance of a layout-constraint engine to achieve it. Do you think you can write something like House of Leaves purely using design tools like Adobe Illustrator? ;)

flexbox sort of invalidates most of your argument

Not at all. No matter how much power you have over HTML elements using flexbox, HTML still has the ultimate say over how the document is split into HTML elements.

In a desktop-publishing program, you can just highlight some text in the linked source, and make it bold, and the layout file will remember that that part of that paragraph should now be bold (using a targeting heuristic that will mostly work unless the paragraph is severely changed.)

You can't do that with CSS.

Well you could mark every div with an individual tag, and then have a realllllyyyyy big css file. ^^'

> bUt MuH TypESeTtInG!

I didn't read the rest of your post after this, because the only purpose of this is to inflame and troll.

The "Website" point brings up an important point.

LaTeX is suprisingly terrible at producing accessible documents. There are various systems and packages which make heroic efforts to make accessible output, but you have to use them from the start, as they work like you suggest -- they reimplement (or at least heavily modify) various LaTeX packages to make their outputs accessible.

I've had a blind PhD student and the standard of using LaTeX to produce PDFs for academic papers has been the single biggest block in his progress.

I assumed that if ctrl-F works, then screen readers also work. Can you share a bit of information about what tends to go wrong, and/or how document authors can fix it?

Almost no documents properly support ctrl-F. You'll get at best single word search, but you won't find coherent sentences. There are arbitrary line-breaks caused by various styling and layout packages (most columns are broken). There are even publishers that purposefully put garbage into the PDF's text layer as a braindead form of "piracy protection".

PDF and TeX are broken beyond repair in terms of accessibility. The former because it allows the text and image to be split, and very few people care about text as long as it looks alright. And the latter because it's entire document representation model is based around computation. Knuth purposefully made it Turing complete in a misguided attempt at timelessness (no need for a replacement if it's infinitely extensible right?), which means there is no proper way to view a TeX document as "just text data".

I've worked in a library digitising documents, and whenever we encountered PDFs we'd just OCR the damn thing.

We need to throw the thing out, swallow the bitter pill of sub optimal kerning, and replace it with HTML asap.

> PDF and TeX are broken beyond repair in terms of accessibility.

All that needs to be done is to have a accessible text section of the PDF--then Tex can just include the source of the document minus the Tex commands into the PDF, and PDF readers can have the screen reader work. The text could also just be stored per page, if the blind require to communicate where they found a particular item to a sighted person.

As I already wrote.

PDF already has exactly that. They separate they have a view layer and a text layer.

But it simply doesn't work, because people can't be bothered to get the text right, once the view layer is acceptable.

Also TeX minus the tags, is not the final text in any way, because it's not markup, it's code.

> But it simply doesn't work, because people can't be bothered to get the text right,

That's hardly "it's broken beyond compare." I think with accessibility formats which represent fully typeset written documents (as opposed to pre-typesetting, like Latex), it's always going to be an issue because they're mainly produced for people who are sighted initially.

We should simply start shaming programs which export to PDF wrongly, it's not really a difficult thing to get right.

"The way a tool is used is the way that is encouraged by the tool."

Knuth is one of the most perfectionist computer scientists out there, and he still got it wrong.

The market economics are simply not in the favour of people doing extra work, and no amount of shaming will get that to change.

The alternative needs to be designed with accessibility builtin, and with decidability/markup as a conscious design choice.

We have such a format, HTML. The chances to get academics to use HTML with the coming open access wave are much better than getting them to rewrite all of their TeX templates.

I disagree, I mean HTML would be even worse for text-flow if you rendered a PDF directly to HTML. The text-flow of HTML is not guaranteed, in the same way that the PDF text section is not guaranteed to be correct.

It's more about the social economics --but if you can find me a tool that supports the PDF export wrongly, I'll fix it myself.

> The market economics are simply not in the favour of people doing extra work,

Most of the tools are open-source. Does Word or any of the closed source tools do a bad job of this? That would be the main stopping point I guess.

It's still super trivial for them to implement, and shaming them will get their PR team evaluating the risk of not implementing such a feature. You just have to be successful at shaming them.

An HTML file also doesn't represent a document on paper, most people writing papers wouldn't switch to a electronic-only format at the moment--each page is carefully typeset and designed, HTML is not at this level even today.

Minus the tex commands doesnt work for maths and tables, they are almost all tex commands.

Let's take an example:

https://arxiv.org/pdf/1606.06389.pdf

A cursory inspection seems to indicate ctrl-F is fine here? Or am I missing something?

Single column layouts often work semi-acceptably for text only, when not purposefully tampered with.

However if you try to select the text on page 2, you'll see that the layout engine didn't actually place the formula into its own section. Not only is it broken, with the sum signs missing, it is also jumbled up with the text below it, resulting in weird - text broken formula - text - broken formula - mumbo jumbo.

This would make it extremely hard if not impossible to properly follow the paper with a screen reader.

The figures on page 6 also produce weird artefacts on the text layer. In HTML for example you'd simply present the alt-text to the person with the screen reader, but TeX doesn't have such a feature in most vector drawing libs afaik.

The table on page 8 is also not correct in terms of the text layer. If you select it you can see that you get the top half of the leftmost column followed by the rightmost column, followed by the rest in some order.

If I may also direct your attention to this gem on page 9: "Then at least half of allnodeshaveh ̄=lgn =lgn−1(seethepaperforthedefinitionofh ̄),and 2√ thus a potential of Ω( lg n). Therefore, the total heap potential would be lg n). Conjecture: the same construction works for the fancy potential √ √ function, which would give a bound of Θ(n · 4 lg lg n)."

Shall I go on ^^?

No need to go on, that's already more than I bargained for, thank you so much for taking the time to respond :)

I suspect we're using different PDF viewers; I'm using the one that comes with my browser.

The results are somewhat less disastrous than you describe, but still bad. (I'm seeing maybe half the problems you mentioned.) I'm a bit curious which viewer you're using.

I can describe what happens when I copy/paste the stuff you mentioned on pages 2/8/etc., and if you're interested I will, but if you're not interested let me ask a slightly different question:

Rather than try to get the screen reader to make sense of the final PDF, would it be easier to just download the original page source from arxiv and let the screen reader deal with that?

I was using the builtin viewer of chrome I think, could also have been safari.

Using the original TeX source for the screen reader is significantly hindered by the fact that TeX isn't a markup-, but a programming language.

TeX is turing-complete by design, making it infinitely extensible, in order to avoid knuth ever having to re-typeset his books. After all if it's a programming language and a new system, problem, style, e.t.c, comes along you can just write a program that deals with it.

But this has horrible effects on render-ability, in order to know what the final document should look like, you need to run it, no way around that, thanks to Rice's theorem.

LaTeX users also generate a lot of their figures with TeX itself, write their own styling or bibliography rules, and write their own custom graphics rendering libraries.

The easiest path is to just give up, render the entire PDF to a 300dpi lossless image, and throw it into an OCR engine. These things contain a buttload of heuristics to generate structure and meaningful text based on visuals, and since we know that humans explicitly (and often only) care about those in TeX documents...

It's pretty darn sad, because in many ways TeX is holding scientific advances back, by eating its own children. My guess is that if scientific papers had branched off of plaintext tools like troff, we'd probably publish papers as machine readable semantically annotated knowledge-graphs by now.

I just tried in Chrome; at least for page 2 it actually did a bit better than my browser: it actually managed to preserve the line breaks! (I don't have Safari installed.)

(Also, if you look closely, the summation signs are not gone, they are replaced by the letter P. Which is not helpful I admit.)

Do you think there's any place here for education/advocacy? For instance, everyone who makes web pages knows to provide alt text for images.

If there was a standard package that everyone knew they had to include or else it breaks everything from ctrl-F to copy/paste to screen readers, presumably people would use it, right?

I'm less interested in speculating what would have been if troff had "won", (though it is indeed fun to speculate), and more interested in how to fix the mess we're in now, so that 10 years in the future, blind people have better choices than OCR.

(Though OCR is still an improvement over the best option in the 1980s I bet. Though I wasn't around so just guessing.)

"Do you think there's any place here for education/advocacy? For instance, everyone who makes web pages knows to provide alt text for images."

That's not a culture thing, this is by the mechanism of <img alt. You won't get people who write TeX to change all of their workflows, and even if there was such a culture.

"If there was a standard package that everyone knew they had to include or else it breaks everything from ctrl-F to copy/paste to screen readers, presumably people would use it, right?"

Then it would still be nigh impossible because TeX commands, like all programming languages, compose rather poorly. It would be a herculean effort to produce a kinda but not really TeX that is both accessible with a focus on semantics, yet still compatible with the billions of lines of LaTeX/TeX out there.

"I'm less interested in speculating what would have been if troff had "won", (though it is indeed fun to speculate), and more interested in how to fix the mess we're in now, so that 10 years in the future, blind people have better choices than OCR."

Boycott LaTeX/TeX and PDF everywhere you can. Whenever you publish a paper, also publish it in markdown/html. Publish in OpenAccess Journals like [PeerJ](https://peerj.com/) which convert all of their papers to html in addition to pdf. Consider publishing papers in alternative forms like nextjournal.com .

We need to get our priorities straight in academia :/. This entire "but latex produces such beautiful documents", "I'm working towards getting into the most prestigious journal" culture of snobbery and vanity needs to stop. We need to go back to caring about the content, not the presentation, something TeX ironically was meant to do.

So basically, give up on LaTeX to PDF as the primary workflow. Either convert LaTeX to semantic HTML instead of to PDF (probably doable for simple cases... \emph to <em>, \section to <h2>, \begin{tabular) to <table>, and so on), or better yet just author HTML without going through LaTeX at all. If needed, convert the semantic HTML to PDF as well, for printing or whatever, and then the PDF might end up sane.

Fine, fair enough. If fixing TeX is hopeless, then so be it. I assumed it just needed a few small tweaks, maybe combined with slightly cleverer PDF viewers. Guess I was wrong.

But then what should people use for math? I suppose there's MathJax, which seems to have put some thought into accessibility.

There's still a problem though. I can't help but notice that the journal you linked to is a biology journal. In some math/CS circles which are TeX's "home turf", TeX is far more entrenched to the point where I'm not sure such things even exist. For instance, arxiv sort of supports HTML, but not really:

https://arxiv.org/help/submit_html

So there I guess step 1 is to make HTML a viable option.

PeerJ has actually quite a few publications, one of them is CS ;) https://peerj.com/computer-science/ the dropdown on the top left allows you to switch between them.

I think MathJax is certainly a step in the right direction, they even support rendering to MathML.

But I agree that there is a certain lack there in terms of full semantic representations. MathJax is more accessible than TeX but it's still describing visual layout, instead of semantic meaning.

Pushing HTML to arxiv is also a step into the right direction.

I think the most important thing we can do is not be complacent with the state of the art. We need to go back to an age of computing where we didn't think we had it all figured out. We need to experiment, and not be afraid to take a step back in some aspects, like layout and kerning, in exchange for other advances like semantic representations and knowledge representation.

I think bred victor has a great talk on this: https://www.youtube.com/watch?v=8pTEmbeENF4

I think we need to experiment with things like observablehq.com or nextjournal.com or the many other that are coming into existence.

re: PeerJ: I missed that, nice!

re: semantics vs visual layout of math... Wikipedia says OpenMath is a thing, but... that only solves half the problem. Once you have a format that encodes what you want, someone has to actually it.

Like, if some writes x^{-1} and f^{-1}, it's hard for a computer to figure out that the first one means "the number you get when you divide 1 by x", whereas the second one means "the function you get when you compute the inverse of f".

And if the author can't be bothered to slow down and say which is which, then the reader will have to guess.

re: HTML to arxiv: not ready for prime time, if you actually follow that link.

re: kerning: TeX's advantage here is not fundamental, I think. Just need a good font, as far as I know. (Actually that's not far; I know almost nothing here.)

re: layout: CSS is finally getting good at this from what I hear.

re: talk: looks familiar; maybe I should re-watch it.

Try copying across a line break. You'll notice that the spaces between the words are missing when you paste it.

Just tried copying across a line break on page 9, seems to work:

Here the potential function is, to say the least, not very simple.

I used LyX to write a pretty significant paper back in 1.6, a decade later Lyx is now on 2.3; I just installed it and opened my old research paper. Obviously, I haven't done a thorough test, but from 5 minutes poking around there's a huge improvement in appearance, documentation, and usability!

There's a similar story for the electronics design software package KiCAD - I played with that a bit in school as well, when it was more like a group of separate software packages duct-taped together with some import/export scripts. The enthusiasts weren't bothered by and seemed to actually enjoy the lack of polish. Recently, though, CERN has injected a bunch of money, programming time, and real-world use engineer's time into the project and it really shows.

I definitely agree that the NSF (or any major LaTeX-using organization) injecting a few million dollars into LyX would be an incredible boon to human productivity. I do feel that it would be better for an organization with internal consumers of the software to undertake this effort than for an outside team to just start with a blank canvas.

The latter too frequently ends up with the primary users being developers, where bug reports or feature requests will be responded to with "Just compile with -D FEATURE_ENABLE" or "For the syntax of that menu item, read the block comment and source code of the foo.cpp/bar() method" - basically, where TeX is right now. Most users of most software will install it with the Windows .exe, a few will use the .dmg/.deb/.rpm or their package manager: a vanishingly small subset are compiling it from source. And if you expect that your users should be able to read C++, your user base is going to very small.

Writing the paper is the easy part, after spending months researching past literature, coming up with models and hypothesis, and performing experiments.

I really doubt Latex is the bottleneck for doing research.

You'd be surprised.

I've spend hours debugging other peoples references, bibliographies and journal dictated styles.

Also if you think that all of academia has MONTHS to work on papers, with our current publish or perish culture, you're wayyyyyy off.

I admit I've been lucky with deadlines. The shortest span has been 3 weeks, but that was one time where most of the work was already done, and only then we looked for suitable conferences to submit the paper.

Nothing in my comment suggests a bottleneck. It's just a matter of making people a few percentage point more efficient at a very low price.

You referred to it as "the single most cost effective way to speed up math and physics research". I don't think it's so surprising that people are interpreting that to mean you think LaTeX is a bottleneck in the research process.

I guess you intended emphasis on the "cost effective" part of the statement, but not everyone will read it like that (I didn't).

I mean, it wasn't merely emphasis. Likewise, "NBA" is not mere emphasis in the statement "Muggsy Bogues is the shortest person in NBA history".

I wish I could upvote this a thousand times and then some.

Already in the 80's Jeff Bezos refused to use latex due to its absurd difficulty to use. I can't believe it's been 40 years and there still isn't a better tool to write papers with.

It took me less than an hour to create a Google Docs template that looks roughly as good as a standard Latex document. The reason Latex looks beautiful is due to a set of wisely chosen defaults, nothing more.

Yet the focus of Google Docs clearly isn't scientific papers. I don't imagine it would take more than a few passionate employees at Google to expand Docs to a powerhouse of scientific writing.

Pictures of my template can be found below:

https://imgur.com/a/cC0e0gG

(and a blogpost about my Latex induced misery) http://mathiasbonde.com/?p=72

I agree beautiful defaults help LaTeX, all the more so because I always use someone else's well defined template. But, my main issues with Docs/Word is not just "look", rather:

- smart figure/caption/table handling (no orphaned captions, no figures hanging out in the middle of the page) - easy, label-based cross referencing (not selecting each from a popup interface) - text-based equation input

All that being said, I still have an elaborate Word template I use for proposal writing. Because often I have to pass it off to others (who may not know TeX) to contribute, and sometimes I need to quickly bastardize my nice formatting to squeeze into hard page limits.

ShareLatex / Overleaf is very good, but I'd love to see slightly more ease of use as OP suggests. I'd like to see a switchable WYSIWYG / code editor, so laypeople can make simple formatting changes visually without needing to know the right commands, and it propagates correctly into the underlying LaTeX.

Totally agree with your points. Latex remains the best tool for many tasks.

But none of the things you described, I think latex does particularly well either. Better than Google docs and the likes, sure, but I think it's possible to do better.

I truly think Google Docs could be so much more than it currently is.

They for example built a brilliant speech-to-text feature that is only available on desktop. Mobile is the one place I could imagine using such a feature. Imagine writing your documents with speech while hiking deep in the woods. Feeling the fresh air with a view of a beautiful brook as Google transcribes your thoughts. That's how I want to write!

Why on earth put so much effort into such a brilliant feature, and then have it only be available to users already sitting in front of a keyboard!?

Apologies for the tangent, it was where my mind wanted me to go!

Just read your rant/blogpost. You didn't give too many examples of what's wrong with LaTeX, but you did mention putting two images next to each other. Isn't this just:

\includegraphics{img1} \includegraphics{img2}

Or is that not what you meant?

Subcaption package https://en.wikibooks.org/wiki/LaTeX/Floats,_Figures_and_Capt...

Okay, nice, but maybe overkill, unless you want to refer to them separately within the main text?

If the images are not related to each other, but you want to place them on top of the page together for stylistic reasons, you probably want a simple minipage environment with two figure environments inside.

I meant they might be so closely related that you want them share the same caption and figure number and everything.

Subcaption works perfectly for that. You can skip the subfigure caption - "The gull" etc in this picture

https://commons.wikimedia.org/wiki/File:Latex_example_subfig...

You can reference either the whole figure by using the label inside the figure environment, or you can reference the subfigures using labels inside the subfigure environments.

> Subcaption works perfectly for that

I'm not saying it doesn't work. I'm saying that if your needs are simple enough, you can get away without it.

The automatic microtype is the killer feature for my own `unprofessional' publishing without Adobe suite.

Besides, the macro language of TeX is not the best programming language IMHO, but it just works, especially for the flexible control of long document.

You can use VSCode as LaTeX IDE and git to sync to overleaf... Which costs slightly less than $20M

My thoughts exactly. I'm currently working on a paper together with an older professor who barely manages to use Overleaf, and this is how I make it work. I keep my copy of the paper on my personal remote git repo. On my local workspace, I keep a clone of that repo with an additional branch tracking the Overleaf version. Then it's just a matter of merging branches whenever any of us make changes. She gets to enjoy the convenience of using Overleaf and I get to keep using VSCode and all my extensions without hassle.

This is going to be really unpopular but you could allow people to write their papers in Microsoft word .

Either that or the other answer would be to say that’s why you have grad students .

It is like asking developers to write html pages and code in Word.

> This is going to be really unpopular but you could allow people to write their papers in Microsoft word .

Many journals do allow Word submissions. Nature, for example[0]

[0] "we strongly encourage you to incorporate the manuscript text and figures into a single PDF or Microsoft Word file" https://www.nature.com/nature/for-authors/initial-submission

Word with a template is the easiest way I've found to write scientific papers.

Word is the preferred format in many fields.

In my field, Word is preferred for conference submissions, because the tight page limits (2-3 pages) is easier to work with in a WYSIWYG editor.

Proprietary payware that doesn't support Linux and doesn't work with Git?

For current users of LaTeX, it's dead in the water.

> other answer would be to say that’s why you have grad students .

Of which discipline?

Any of them. Professors generally chase grants to fund grad students who need tasks to get a PhD.

A lot of journal and conferences accept Word file (and provide MS Word template). My dad published tens of papers per year in biology and he doesn't even know what LaTeX is. However, for Maths/Physics/CS, a lot of people just find LaTeX to be faster overall (me included, I began using LaTeX simply because I was too tired to typeset my equation in MS Word)

I work in a lab which publishes in both CS and biology journals. Biologists require Word. Most our people hate it more than LaTeX, since in LaTeX it is at least difficult to break the template accidentally. Often the writing starts in GDocs, then gets moved to LaTeX, then converted to Word and cleaned up manually.

I'd love Word if it had that one feature, disable direct formatting, to enable changing formatting only through styles.

BTW Latex with embedded Markdown is a rather pleasant combo after you have it set up properly.

Word has that feature. Type "Protect Document" into the search bar, then select "limit formatting to a selection of styles" in the options toolbar that shows up on the right hand.

I don't trust articles written in Word.

I have an alternative view: The fastest way to speed up research would be to fund an open source Wolfram software suite with feature parity. For a lot of hard tech you need good multiphysics and CFD simulations, both of which suffer from a severe lack of open source tooling. Most of the proprietary engineering software can be cloned rapidly once a solid foundation is built. Even though Julia and Python are making progress, in terms of ergonomics and productivity it is still hard to beat the offerings by Wolfram.

(If somebody is looking for a project for Mozilla's "fixing the internet" incubator, maybe give this a shot.)

Python ecosystem is not even close. I recently switched from Mathematica to Python, and I am going mad from juggling half a dozen packages with overlapping features.

Really wish somebody would make a meta package consisting of numpy, scipy, sympy, sage that removes conflicts and overlaps, and renames all functions with the same naming scheme.

I am surprised nobody here mentioned TeXmacs. It's nothing to do with TeX or Emacs, but it's inspired by them. Basically the concept is structural graph based document editing software similar to Tioga/Cedar by Xerox back in 1980s but unfortunately the research and improvement for better tools for editing document seems to stagnate a lot after 90s. I think the idea of having an easy to use WYSIWYG editor for scientific document editing need to be popular now to significantly increase scientist productivity. Remember that web was invented to increase scientist productivity and the rest is history.

On a related note, personally I think Overleaf or Google Docs is missing the point or the pain points. Even though both of them enabled offline editing but they are really a cloud first applications. The better approach is to have something like TeXmacs (native desktop first application) and make a seamless integration with synchronization and versioning to the cloud. With a proper use of synchronization and versioning technology like rsync and Git, together with seamless connectivity tools and protocols for examples Wireguard VPN, WebDAV or even the latest SMB over QUIC approach, I really think this is very feasible. The CONCEPT is similar to the useful and successful Watcom product or now SQL Anywhere database editing application.

I'd say, make sure it's backwards compatible with the old way of doing it because there's a lot of scientists who have been using it for a looong time and don't like change. You have to remember, for a scientist, a tool is just a tool, not all of them are going to invest a lot of time in learning a whole new process.

The old way doesn't have to change. It still exists. Make a wrapper GUI that makes it far more approachable. Syncing and other funtions could be handled by a library that works with either

Oh, it is a lot of effort. Just pouring a lot of money and hoping that we can get through warts (without losing the best parts) of LaTeX is unreasonable.

And in fact, there are projects:

- https://www.mathjax.org/, and its much faster (and less feature-complete) variant https://katex.org/

- http://www.luatex.org/ - I guess this is the closest to what you mean

- https://rstudio.github.io/distill/ (using Knitr; it can generate HTML or PDFs)

etc, etc.

Even now, other libraries use BibTeX, which would be the easiest to replace in some more standardized format (JSON or YAML, so it is easy to load from web, directly).

No need (edit: except maybe for collaborative editing). Use TeXLive and install the updated copy every year (it has annual releases, and you simply delete the old install directory).

If you're not on linux, spin a Ubuntu VM, and install TeXLive.

(Strictly speaking you don't even need GUI on the machine where pdf gets compiled. I add 'scp output.pdf mydesktop:.' to my Makefile so the file is transferred to my desktop everytime I make. On the desktop the pdf is already open on the big screen and if your pdf reader is good, it'll update the opened pdf everytime the file on harddisk gets updated)

I never use overleaf, and I had no problem finished up my thesis in record time. LaTeX wasn't even an issue.

edit: Overleaf will be helpful for collaborative editing though. I personally would use github/gitlab instead if I ever have to do collaboration.

Seems easier to take org or markdown and add stuff to it until it's on par with LaTeX.

Markdown is not even 10% of the way toward a full document typesetting system, so you might as well throw it away and create a new codebase. The only thing you're really suggesting then is making it backward-compatible with the markdown syntax.

Is markdown anything other than syntax? But yes quite a big part on (mostly) pagination is missing, which will need to be added.

I still think this is easier than merging the commonly used LaTeX packages into one code base and then building a proper IDE and a method to output to HTML.

Much better to start with markdown, extend KaTeX to support whatever you still need for math notation, and then work on making pandoc output to PDF (or equivalent) natively.

On of the bigger problems is bridging the gap between unpaginated HTML and paginated PDF. However if you start from the LaTeX side you'll need to deal with decades of technological debt as well.

Actually the monolithic Tex has been done. Its called context. (Every couple of years I try for a few hours to install it under windows but then give up). Sometimes I manage to install it and then give up because of the introductory texts.

https://en.wikipedia.org/wiki/ConTeXt

(but I still believe there is gold at the bottom of this)

Edit: woooah. I see there is now an installer for context, since april 2019! https://wiki.contextgarden.net/Installation

here we go again....

Using a scripting language to format text always seemed insane to me. People love LaTeX but it's nowhere near as intuitive as writing formulas down on a piece of paper. Having two panes, one for the output seems also very hacky and unintuitive.

Instant feedback in one view, typing at the speed of thought should be the goal, and if it means a new paradigm I'm all for it. Imagine it was something you could actually program with; those gross inline math functions would suddenly be so much more parsable.

LuaLaTeX ?

Remember that scene in the matrix when he says "I don't even see the code anymore". After a short while (about a couple of hundred pages of typing latex math for me), the code is mostly as intuitive as the visual formulas. At least for not too long equations.

[1] https://www.youtube.com/watch?v=MvEXkd3O2ow

LyX exists. It is used only by a small minority of physicists even though it's been around for 25 years.

You should check out Tectonic.

https://tectonic-typesetting.github.io

It fixes some of the issues that LaTex has.

The important/difficult problem of generating beautiful web pages is most of the way there already: https://github.com/arxiv-vanity/engrafo

There are some features in LaTeXML, which powers Engrafo, for adding some "LaTeX++" features, like embedding JS, etc.

Still could do with a lot of work, though.

Yes for IDE. Yes for HTML+ output (though what exactly the "+" entails is a tricky thing; HTML+CSS won't suffice). Not sure about merging the 100 most popular packages. Most likely their conflicts cannot be resolved in a backwards-compatible way, and then you have the standard "101 standards" outcome and old tex files start rotting.

Yea, I really meant merging in the functionality of the most popular packages, and keeping as much of the syntax as possible without compatibility issues. I agree with you about those risks.

Imo papers of the future should be authored in semantic HTML, which can be rendered to PDF, but most importantly be reflowed and read on any digital device, also by people with disabilities, and indexed/searched.

The capabilities are all there - MathML, CSS for print layout, <figure>, SVG, highlight.js, LaTeX.css, ...

It's the right platform, but it's the wrong authoring experience for people who might not be able to code at all, much less understand the nuances of CSS and how it interacts with the DOM. People using LaTeX don't want to code something, they want to write something and just have requirements that go beyond the typical use-case for "writing something". A layer is needed on top.

Maybe that could take the form of a single JS and/or CSS file that gets dropped into an HTML page and then the author can create their documents with nothing but special tag attributes/classes. But likely it would need to be even more abstracted than that.

Here's a good attempt that's being made. It's an extended version of Markdown that covers charts, equations, etc.: https://casual-effects.com/markdeep/

I disagree (or maybe because I don't fully understand your comment). In my (limited, I'm not a FE dev) experience, you layout an HTML page by hand but I do not want that with a research paper.

I want to write my paper and let LaTeX lay it out for me. Of course I'll have to wrestle with an unruly table or figure now and again but for the most part, the layout and typesetting of the document is handled for me.

I wrote some documentation in html and wanted to export it as pdf. The main blocker: an index with page numbers. css has them, but support for them is mostly non existant.

Luckily Knuth himself has this on his radar as this "earthshaking anouncment" by him shows: https://youtu.be/eKaI78K_rgA (Won't spoil it by giving more details)

Don't most journals accept MS Word submissions now? Doesn't Word provide all the formatting needed?

I also prefer Latex but I wonder if we're talking about solving a problem nobody has?

Interesting. When I did some internships at Los Alamos (LANL) my lab neighbor was a theoretical physicist. He used Word and referred to LaTex as the Fortran of word processors.

How about a modern TeX formatting layer to replace LaTeX?

This sounds like an opportunity for someone to create a python module to generate scientific documents.

You’ll get all the benefits of python, with the added benefits of generating formatted text.

Call it pyTeX.

LOL..

Reply on news.ycombinator.com