Author claims that the most likely is that there is Tesseract running behind ChatGPT-4v/o.
There is no way that this is Tesseract.
-> Tesseract accuracy is very low, it can barely do OCR on printed documents.
There is no way that this is Tesseract.
-> Tesseract accuracy is very low, it can barely do OCR on printed documents.
For example, GPT4 with some vision capability would be able to fill in the incorrect OCR with the additional word co-occurrence understanding.
I've tested this approach with purely text LLM to correct OCR mistakes and it works quite well.
Also note that in some newer OCR pipelines that don't involve LLMs, there is a vision component and then a text correcting model that is in some ways similar to some forms of spell check, which can further improve results.
I doubt they are running an OCR model, but if they actually were it would likely be an in-house one trained with more modern techniques.
multimodal llm would of course blow it all out the water, so some llama3-like model is probably SOTA in terms of what you can run yourself. something like https://huggingface.co/blog/idefics2