Hacker News Reader

Comment by kuschku | Hacker News Reader

Based on the video it works amazingly well. I’ve tried finding a usable way for open source text recognition on desktop, and haven’t found anything even close to this.

Like, I've got scans of letters, machine-written, 300dpi, perfect black/white 1-bit depth, no marks or scratches, perfect quality — and OCRmyPDF (using tesseract) absolutely fails at this task, returning only bullshit. Even if I set the language correctly, or even set the wordlist to a manual transcription of the PDF.

I also tried using OCR on screenshots, with the same miserable result.

How does Apple’s Vision API do so much better at this kind of task? Is there some trick I'm missing?

Like, the images I supply are of such high quality that you can literally just split into characters and search for the nearest match in a database of latin characters, and even that would return better results than tesseract.

View on HN

I think you have to fiddle with the internal settings of the OCR package you are using to get good results. For tesseract and pytesseract, there is a whole article on improving the quality: https://github.com/tesseract-ocr/tesseract/wiki/ImproveQuali...

My guess is Apple trained on their own massive dataset and better architectures that make their systems better than off the shelf ocr.

I am working in this area and one way I see good improvement is training an object detector to first detect words in an image, then you can pass that through tesseract/OCR software. Besides that, finetuning tesseract on data you want strong performance would be the next best alternative.

Here is a cool article from dropbox how they engineering their OCR system: https://blogs.dropbox.com/tech/2017/04/creating-a-modern-ocr...

This link below is a really cool project that showcases what building your own OCR engine would look like!

https://github.com/awslabs/handwritten-text-recognition-for-...

Reading all these comments about others having bad experiences with Tesseract makes me think there is a market for a robust, better alternative to Tesseract.

In my limited comparison. Apple’s API performed way better than Tessaract. You can check your images using this app or their sample xcode project for vision, I think same API works on macOS too.

I would also vouch for google’s ocr API but I opted for Apple’s for privacy reasons and as they were easier to implement.

> but I opted for Apple’s for privacy reasons and as they were easier to implement.

Thank you for respecting user privacy and making it a first class citizen.

I use Tesseract to extract text from my scanned documents in order to be able to grep through them. What surprised me, was that the lower quality (resolution) scans had actually much better OCR results. So it seems Tesseract works best with some specific font-size-DPI combination.

This is the script I wrote for that purpose (maybe it helps): https://nopaste.xyz/?ea1e6242a659ce5c#vAMyPpajo0OIeCGQe9Wkdq...

The results aren't perfect, but good enough for my purpose.

You get what you pay for with Tesseract, don't judge OCR performance based on it. Would highly recommend checking out FineReader if you're on a supported platform. You can feed it crumpled beer receipts and the results are still decent

If it makes you feel any better, I had a similarly terrible experience with Tesseract. I spent days studying the docs and I failed to find a setup where it would produce anything useful even with the most perfect input.

I was also left with a feeling that I must have been missing some trick, because people on the web seem to be using it with good results.

It requires a lot of tuning. When I first tried it out I rapidly abandoned it when I couldn't get useful stuff out of a screenshot of text I typed in.

I don't know if this is how it works or not, but it is worth noting that searching for text in images is a much easier problem than transcribing images, and if you do the latter as "step 1" you destroy your ability to do the former: if you search for some piece of text you want to find all images that maybe if you squint a bit could be that piece of text, which allows you to deal with cases where your recognition is forced to make a bad guess that does the moral equivalent of rebracketing symbols that is difficult to work around in a later text-only search pass. If you can at all avoid it, always avoid transforming your input information until you know your query, lest you destroy information (this is what always made Microsoft Journal so epic for me, though I should be clear that I haven't used it in 15 years so for all I know they forgot this).

I had a similar experience working with OCR for a similar project as OP ( https://www.askgoose.com ). What we realized was tesseract's training data was significantly different to what we were trying to transcribe (screenshots of conversations). The sparseness of text leads to bad transcriptions. One workaround is to do some image processing before hand so you can break the image down into data that looks similar to the train data used for tesseract.

I've had success with first using imagemagick to cleanup the image, do some thresholding and then using the latest version (4.1) of tesseract. With tesseract you can also try to experiment with different --psm values for better results.

It might be worth trying this for desktop: https://github.com/the-paperless-project/paperless

It handles OCR, indexing and search

It appears to also use Tesseract

The paperless project has some tweaks in the config that can get decent results out of the box. For general use with decent quality documents it will work pretty well right now versus spending hours tweaking Tesseract.

Tesseract needs text on a white background to work well out of the box. For "text on images" try for example https://ocr.space/ocrapi

You might like Screenotate by Omar Rizwan - https://screenotate.com/

Is there a way to port Apple's Vision API to javascript for a webapp, or use their models without an Apple device like Apple Maps on Duckduckgo?

I don't think that's possible. The Vision VNRecognizeTextRequest is only available for:

iOS 13.0+ macOS 10.15+ Mac Catalyst 13.0+ tvOS 13.0+

https://developer.apple.com/documentation/vision/vnrecognize...

Reply on news.ycombinator.com