Image to LaTeX | Hacker News Reader

Image to LaTeX

wandb.com

121 pointscvphelps37 comments

So this is just like Mathpix?

I’ve never found typing out LaTeX equations particularly tedious, unless you’re doing multi-page derivations in which case you’re spending more time checking for errors.

Yeah, I've been using Mathpix for almost a year now and this seems practically the same.

Unless they are deliberately avoiding comparison to Mathpix, they are quite unaware of the competition out there :|

We were aware of Mathpix and we think it’s an awesome tool! We didn’t mention it because there isn’t any publicly available performance results for the model they use, so we weren’t able to compare it to ours in a quantitative way :)

But you could do the performance evaluation of mathpix using your benchmark or?

The evaluation sets involves ~20k samples. They only let you do like 100 for free. Also I’m not sure there’s an API for it for me to do automatically for such a large scale

Off-topic, on Mathpix' site, the image of a Mac contains a battery icon. Didn't know they now make Mac's with battery backups :)

Pretty sure Apple made the same mistake on iMac boxes.

They aren't tedious until you try to do large matrices where you need to use ellipses and other things inside.

Also Latex notation for even simple things like paretheses sucks. \left( and \right) ? wtf? why not just \( and \)? the left and right is redundant. Stuff like that makes it unpleasant to use.

They do different things, ( and ) work just fine, but they are standard sized, while \left( and \right) adapt to the size of what's inside. It's kinda uncomfortable to write, but I guess it is done this way because the \left and \right commands can take a lot of different arguments

The stated use case is to allow scientists to quickly add equations from other sources into their papers. However, if you look at output 2, there are some subtle mistakes in the result. It seems like whatever time is saved on transcribing equations would be lost on proof reading them.

It does seem like this could have value as a LaTeX teaching tool though, for when you need to figure out how to write the markup to do a certain technique.

Yeah, for my domian I just download the latex source from arxiv and take equations from there if there is any point at all to not just transcribing.

Unfortunately, typesetting the equations directly is often the wrong way. It might sometimes work for lecture notes, but when you develop your own scientific work, your notation is often still fluid. In such a case it is sensible to express your mathematics with a domain-specific notation. Then you can later a) change your notation and b) understand your source code.

As an example, instead of writing \(G = (V,E)\) directly you might want to write \(\graph{} = (\nodes{},\edges{})\)

> In such a case it is sensible to express your mathematics with a domain-specific notation.

> As an example, instead of writing \(G = (V,E)\) directly you might want to write \(\graph{} = (\nodes{},\edges{})\)

This can be great for personal use, but it can be very bad for communicating with others; (a) you won't be able to write a paper with a co-author unless you agree on your domain-specific abbreviations in advance (and, if you think that's easy, then I say from experience that it isn't), and (b) most journals prefer a minimal number of custom commands, and some forbid custom commands entirely. Even if journals permit custom commands, the more you define, the more likely you are to conflict in some baroque way with their own style files.

EDIT: Also, even if you do this only for personal use, it can be very hard to go back from it (just search for TSE questions on macro-expanders; there are plenty, but all of them that I know of assume some well formedness constraints that make sense to their authors but will almost inevitably conflict with one's own ad hoc conventions).

Eh? I use tons of macros in my papers with 5+ coauthors all the time; it's preferable to random per-person notation that is inconsistent over the paper.

In fact it's the only way to scale a paper across tens of pages and over multiple people

I must say I never had difficulties to get to a common notation. If all authors agree that such a thing matters, that is. If authors don't care much, they are sloppy anyways. I also can't tell much about the ways of journals as I mostly published for conferences that worked with PDFs from a style file.

If this works on handwritten equations it would be great. I used to work at an online school and getting teachers to be able to write latex was next to impossible.

Microsoft is slowly wandering in that direction. They've got the math input panel: https://www.digitalcitizen.life/do-math-easy-way-math-input-...

And they're starting to use LaTeX in office equations: https://support.office.com/en-ie/article/linear-format-equat...

There are quite a few educational apps that does this. E.g. look up socratic.org's GitHub.

Sorry, I don't see it:

https://github.com/socraticorg

Socratic is powered by MathpixOCR :) https://mathpix.com/ocr

Project: http://lstm.seas.harvard.edu/latex

Source code: https://github.com/harvardnlp/im2markup

Reminds me of Mathpix Snap: https://mathpix.com/

Yep! In fact we chose to work on this project after seeing (and using) Mathpix. We thought it solves a very interesting problem using deep learning :)

Another way would be to use the neural network to do segmentation, run the individual results through DeTeXify and finally reassemble. This would probably yield higher accuracy (DeTeXify works with handwritten too).

Interesting idea!

I Like this but I notice that even they examples in the paper are buggy.

But it could be like Google Translate: does a terrible job but makes a handy starting point you can then fix up to be reasonable.

This is awesome I was looking for such a tool. Indeed handwritten seems like the next logical step.

Or scanning older math-y books that have no LaTeX equivalent.

This is only a problem that needs solving because LaTeX formula syntax is pretty horrible.

I'd like to underline that, for all its warts, MS Word has an excellent formula editor (which even supports common LaTeX commands as autocorrect). I know that it's a radical idea, but if the goal is to let scientists spend less time writing formulas, maybe we should consider letting them use modern software?

ps. MS Word can copy formulas into the clipboard as LaTeX expressions, should your journal require LaTeX papers but you dont want to edit your formulas like a caveman.

  > for all its warts, MS Word has an excellent formula editor

This just isn't true. The problem is that the domain is complex, and it is difficult to support both simple and complex expressions well in way with good UX. MS Word's editor is good for simple expressions (although the typesetting is never good, sometimes terrible). For complex things, latex is much nicer.

Honestly, if you are taking the time to write something up properly, the time spent on the equations is a small fraction. And in most contexts it's just more efficient to use LaTeX anyway, at least for papers, manuscripts, thesis etc. So you save more time. If you are collaborating with multiple people you can use a revision control system and have much less pain that track changes, etc. And no more 30s rerendering loop in Word for no obvious reason once you've added some complex content.

More Anectdata: I've watched multiple people give up on writing a thesis or paper in word due to too many issues, and then moved to LaTeX. I've never seen someone go the other way due to limitations they hit in LaTeX.

It's extremely tedious to type formulas in MS Word... an insane amount of clicking around the toolbars and then clicking in boxes. In my experience Latex is far easier.

Further, this tool is designed to turn screenshots taken from papers into LaTeX code. It's faster than any editor because the user does not have to enter the equation at all.

Word equation editor is a useful toy for people that need a couple of equations here and there.

In college I used to transcribe my notes into nice pdfs using LaTeX, at one point I had snippets for all the common equations, I wasn't even thinking about what to do to write it down, it turned into muscle memory.

Besides that, Word now has support for LaTeX, it speaks pretty loudly about LaTeX qualities over the "let me look for the line integral symbol in these menus".

Yeah, it has a pretty steep learning curve, but once you get over it there's no better way to write equations.

Obviously, it is not for everyone, as I said, if all you need is a couple of equations, Word point and click equation editor is more than enough, but there's nothing "cavemanly" about LaTeX, it's powerful and once you learn how to use it it doesn't stand on your way.

> if the goal is to let scientists spend less time writing formulas

People with a few weeks of consistent practice writing LaTeX can run circles around anyone using a graphical equation editor.

I knew students in college who could type LaTeX twice as fast as I could write formulas with pen and paper.

I tend to disagree on this, and offer a different perspective: I believe a markdown-like syntax can be way more efficient and RSI-free than a mouse-based WYSIWYG-like UX.

That said, I think both Latex and WYSIWYG editors tend to lure users into unhealthy habits the moment they get over the basics.

In the case of WYSIWYG - users tend to spend way too much time on fine tuning every single instance of produced output, which is very much labor intensive busywork. Powerpoint is IMO the canonical example to watch this behavior.

In the case of LaTeX/markdown, the somewhat more experienced users get lured into learning more esoteric portions of the underlying system and soon wants to automate everything, by writing TeX or LaTeX macros, hacking emacs modes and whatnot. Arguably, this second mode of wasting time is more intellectually challenging and more fun for some, but maybe just a waste of time still.

If one sticks to standard TeX or LaTeX, one can be fast and produce good (but maybe not super spectacular) output.

> I know that it's a radical idea, but if the goal is to let scientists spend less time writing formulas, maybe we should consider letting them use modern software?

What does 'let' mean here? Who is stopping them? (Except possibly journals that want LaTeX submissions—but, as you point out, Word can export LaTeX equivalents of its formulæ.)

Would you use a flowchart based visual programming tool for general purpose programming?

Scientists do not usually use Word for somewhat similar reasons.

Reply on news.ycombinator.com