Detexify: LaTeX Handwriting Symbol Recognition | Hacker News Reader

Detexify: LaTeX Handwriting Symbol Recognition

detexify.kirelabs.org

173 pointssusam38 comments

Related (but not identical), Facebook research just released an open source pdf -> markdown reader (that does a good job w/ equations in latex).

https://facebookresearch.github.io/nougat/

I've used it to convert 40 page pdfs into text, and it did an impressive job.

I've had really good results with it so far. I'm using it in the Huggingface Transformers library, and it's been great for my workflow.

No one in STEM would get their PhD without this tool. It’s amazing

This is what I consider to be truly useful AI without the buzzwords. Doesn't need those buzzwords either.

Can get rid of the AI buzzword as well then. This is machine learning.

Fun fact: it's written in Haskell

I remember what a great help it was in writing my LaTeX documents back in 2010. Probably the forefather of all these GenAI assistants :D

and

https://lukas-blecher.github.io/LaTeX-OCR/

This is quite nice. The generalization of this to full formulas is much more difficult. In most languages, letters are designed to be distinct, and in case of confusion, many times a secondary pass with a dictionary [1] helps quite a lot.

But mathematics symbols are often quite close, as evidenced by this app. There are multiple suggestions for each input, and they are often quite close that without larger context it is difficult to tell which one the user meant. My handwritten mathematical x, greek chi, and the times symbol look very similar. Alpha and "proportional to". Etc etc. At minimum, the ai would likely need some understanding of the structure of mathematical statements and equations to do this better.

[1] Or even a third pass with a grammar checker.

See

https://facebookresearch.github.io/nougat/

and

https://lukas-blecher.github.io/LaTeX-OCR/

These are on digital documents, where the characters do look different. Handwritten characters look similar. It's a much more difficult task.

There's also an android app! It's nice to draw with your finger instead of a mouse.

I prefer drawing with a pen, then a mouse, and only then my finger. When I lift my finger up and put it back down I don't know the exact point it'll register as!

Sorry to go all Hacker News on you.

Instead of drawing with a single isolated finger, try writing with an invisible pen: pinch your fingers together like you are holding a pen (e.g., index, middle, and thumb), lower it to the screen until one finger touches, and then draw as if you had a pen in your hand.

This lets you use your same muscle memory. No need to guess where your finger will register. I prefer this dramatically more than drawing with a mouse.

that android app has been around for over ten years! close to fifteen maybe!

There is a desktop app version of this, it rocks: https://github.com/zoeyfyi/TeX-Match

I use this every week and have been using it for years. Huge fan.

I once tried to build a replicate with ML classifier. It was quite fun. And to achieve the accuracy of detexify is quite hard

Handy, although gotta say I hate to be that person but GPT 4 is light years better than this tool. You can provide it with a picture of an equation you'd like to obtain as LaTeX commands and it'll do its job. Done it 30+ times so far and it's been 100% accurate.

LaTeX is one of those domains where GPT4 is amazing. Along with eMacs lisp, tho it is better at LaTeX than elisp, where I have had it cycle into non-convergent series of errors. Much better than it is at actually making sense of the mathematics, interestingly.

But GPT-4 isn't open-source.

Seems like a different use case. This tool is very helpful when you know what the symbol looks like but you don’t have an example sitting in front of you.

Curious how they trained it.

A lot of volunteer training data - I contributed a bunch to it when it was under development!

This is pretty old. It's been around for at least a decade, maybe longer. Do you remember when you were working on it?

I don't remember the exact year, but 2009-2010ish I think. Definitely more than a decade ago.

edit: Actually after searching my email, I think I started using it in 2007.

Maybe https://github.com/kirel/detexify-hs-backend/blob/master/src...

https://github.com/kirel/detexify-data

found this site recommended for finding a way to replicate maths equations in your own latex document - https://mathpix.com/

it is very effective for long and complicated equations. unfortunately you need an account to use it.

I remember Maple had this 15+ years ago.

Haven't used this since college, but it was very helpful then!

Can someone explain the trend of ditching SSL certificates?

What trend? It's served over TLS for me.. DV cert from Let's Encrypt.

It might try and load a webfont over http:// but any decent browser will just refuse to load that sub-resource.

Interesting. Thanks for the heads up.

This site is served over TLS for me, and I don't see HTTP-only sites as a trend, but there are a couple reasons to forego SSL:

1. Decentralization

2. Better performance

Opting out of SSL is fine for static sites that don't handle any kind of authentication.

Why is a lack of SSL a form of centralization?

I'm meant things are less centralized without SSL - I worded my other comment wrong, edited now.

Update, some how I commented on the wrong thread.

Reply on news.ycombinator.com