Australia's Trove is getting humans to translate them‽ - https://trove.nla.gov.au/help/become-voluntrove/text-correct...
Anyway, very cool, the world needs more of these out of copyright newspapers online. History has been lockup up by historians for to long.
There is Tesseract: https://github.com/tesseract-ocr/tesseract
> Australia's Trove is getting humans to translate them‽
Your link clearly refers to correcting an existing transcription:
"While viewing digitised newspaper and gazette articles, you may notice that the text transcript doesn’t always match the text in the article. You have the power to fix this by editing the transcript to match the article text."
Most likely they used OCR software to generate the initial transcript, but allow users to correct the OCR output because they know the software is not perfect.
I know there have been improvements, and probably impressive ones, but they are not being implemented in any sort of unison in anything easy to use.
If computers can't even read newspapers, which only have a certain amount of fonts and letters/words to chose from how can we think they can do much else.
Test of - https://trove.nla.gov.au/newspaper/article/151059024
Tesseract starts really well here but then stops after a few lines. No idea why, it just frustrates me.
"FEMALE PIRATE LEADER. HONG KONO, Sunday. After no sign of activity in Bias" - Then Stops.
Trove is below and doesn't get much.
"FEMALE PIRATE LEADER. HOHCrKOiro. SanOV. After twiigiiJofswUTlty In »» Xay far ? jux. pirates, lei br a woman, aged 28, EagUA tpio^K, attacked the JspaBMe steaiktr TM1 Slant ? betwsan jbaojr and"
ABBYY Finereader is OK -
"FEMALE PIRATE LEADER BOKO KOHO, Sunday, After no ties of activity la Bin Bay for a year. pirates. lad by a woman, aged 28. English speaking attacked the Japanese steamer"
https://chris.improbable.org/2014/3/17/content-search-on-a-b...
https://blogs.loc.gov/thesignal/2014/08/making-scanned-conte...
Basically the big problems are getting the content deskewed (even a slight rotation will cause accuracy to plummet, which is a problem if there was page curl or a flaw in the original printing process), breaking text into clean segments (non-trivial in newspaper layouts), and dealing with noise from dust or content from the other side of the page bleeding through. The collection I’ve worked the most with (https://chroniclingamerica.loc.gov/) also had a lot of problems due to many collections having been scanned from microfilm first. Tesseract 4 is better but in my testing you aren’t going to see revolutionary improvements without investing in tooling to identify segments and clean them up before passing them to Tesseract.
Since that entire collection is public domain and freely available for download (https://chroniclingamerica.loc.gov/data/ or s3://ndnp-batches), researchers have used various ML tools on it and that definitely looks promising but is not a silver bullet by any means. There are some trained files available here along with a large public S3 dataset:
The closest I can find is Tesseract which was developed in the '80s, and for handwriting it gets maybe two thirds of the letters right. Doesn't work on cursive.
I want something that gets 99 point something right and works on cursive, that I can use as a library, offline. Plus flying cars, and a pony.