Hacker News Reader

Comment by esafak | Hacker News Reader

Apparently "deep document understanding" refers to OCR and structured document parsing: https://github.com/infiniflow/ragflow/blob/main/deepdoc/READ...

Since "deep document understanding" is not a term of art, I would have just said "OCR and document parsing".

How well does it work? Please include benchmarks. You may be interested in

https://paperswithcode.com/sota/optical-character-recognitio...

https://paperswithcode.com/task/document-layout-analysis

The models seem to be closed source, hosted here: https://huggingface.co/InfiniFlow/deepdoc

View on HN

We've used YOLOv8 as the object detection model, and use some public datasets, such as PubTable, CDLA, together with some private data to train the model. The model on Huggingface is the one trained using public dataset, and we would open this work later. We use YOLOv8 just because we want to let the document parser run without GPU, I think you could also try any other object detection models such as Detectron, and use the public datasets to train the model as well. We've not used transfomers for this task, because given limited data, it could not outperform traditional CNN based models.

A bit off topic but every time I see DTrOCR I remember that marketing is a good idea because DTrOCR[1] and Fuyu[2] are basically the same architecture[3].

[1] https://arxiv.org/pdf/2308.15996v1.pdf [2] https://www.adept.ai/blog/fuyu-8b [3] If you don't want to search for the figures I made a tiny post about it on my weblog: https://weblog.snats.xyz/posts/2024/02/16/

RAGFlow uses Yolov8 for its OCR/layout recognition/TSR(table structure recognition). And RAGFlow uses large amount private data to train these models for them to perform well in some specialized scenarios.

Ok we've taken deep document understanding out of the title above. Thanks!

I am curious about the performance of their OCR and layout and table detection. Hopefully it’s on par with Amazon, Google, or Microsoft’s tools.

Reply on news.ycombinator.com