Real-Time Recognition of Handwritten Chinese Characters
machinelearning.apple.com
machinelearning.apple.com
However, when I studied Chinese for a bit, the (printed) characters are tricky to start with, but reading a fluent writer's handwriting ("unconstrained", as they call it in the article) is nigh impossible.
I think stroke order is hugely important, in that it forms the conventions for fast abbreviated writing. For example, the "mouth" radical, a square, "口", when written quickly basically looks like a lower-case "b", because the left lower stroke comes first, then top and right in one stroke, then the bottom stroke.
More complicated characters basically become a squiggle, and it's really magical, if you think about it, that you can just read it. I mean, look at figures 5, 6, 7 in that article.
For me, a nice illustration of Moravec's paradox of AI [1]: what we think is difficult (chess, differentiation, ...) is easy, and what we think is easy (reading handwriting, putting an egg in the fridge, walking (!) and opening a door [2]) is hard...
[1] https://en.wikipedia.org/wiki/Moravec%27s_paradox
[2] https://www.youtube.com/watch?v=g0TaYhjpOfo ("A Compilation of Robots Falling Down at the [2015] DARPA Robotics Challenge")
It is when you are starting out, but it's not so bad once you get used to it. The best way to learn to read Chinese handwriting is to learn to write Chinese handwriting, and I don't just mean practising writing by hand, but to work through a textbook aimed at teaching cursive Chinese.
This book is a good introduction in English [0], and if you are comfortable with Chinese only, this book is excellent and also far more comprehensive [1].
Once you've gone through those books, you'll be able to read most handwritten stuff without too much struggle.
0: https://www.amazon.com/Chinese-Cursive-Script-Introduction-P...
In this case, I had no idea there was cursive Chinese. I've since gone on a search and learned more. I'd have never done that, had I not seen your comment.
Thanks.
I once worked with a paediatric neurologist whose handwriting was so bad, even he couldn't read it. We had a patient with a referral from him which we couldn't read, took it down the hall to him... he scrunched up his face... and gave up reading it: "Well, I know the patient, and they need...". Absolutely lovely bloke, but I'd hate to get a referral from him without being able to talk to him :)
Yes. All of the 的 examples are clearly 的 that I'd get without being primed.
Figure 7 is more tricky but I'd know 五 without being primed, and 王 definitely after being primed or if I was reading handwriting that had more context.
I once lightheartedly asked my teacher if exercise books in china had bigger gaps between the lines, as I couldn't see how to fit a vertically-complex hanzi in. She wandered over and scrawled something that neatly fit in the line. "How the hell did you do that... ?" :)
A year is probably still too early to be able to read handwriting well. You probably need to have about 2,000 known characters under your belt and also have a good feel for the language such that you are able to fill in gaps based on context - that is be able to complete parts of the sentence based on knowledge of what words/characters are likely to fit.
There are many cursive characters that reduce to something quite similar so having knowledge of what fits in the sentence makes things easier.
As an example, with 王 above, by itself it might not look like anything you can recognise, but if it was written like that on a note starting out like 尊敬的王老师 then from context it narrows down the possibilities significantly.
> She wandered over and scrawled something that neatly fit in the line. "How the hell did you do that... ?" :)
Practice! It's one thing to know how to write a character, it's another thing to have the fine motor skills required to write it proportionally within a confined space, and you can only develop that skill through practice. If you're interested, one of my side projects [0] is a site that lets you generate custom Chinese worksheets of varying grid sizes and fonts (including some handwriting fonts) and other things, which can be useful for such practice.
Thanks for the link - it will help and I've been meaning to look for one (phone apps with your finger aren't quite the same).
'Pretty much every night' is the most important part.
Learning Chinese characters is a grind and you just need to keep chipping away at it.
Pick a nice sustainable number of characters (maybe 6, maybe 3, maybe 10 but something that takes maybe 30 minutes to an hour to do including revision) and then do it every day without fail.
Number of days in a row spent learning is far more important than number of new words learnt per day, so always try to optimise for the former rather than the latter.
At some point, almost every Chinese learner who has spent more than 3 years learning the language has looked back and thought 'why oh why didn't I just learn 3 characters a day'.
It seems so much more difficult if you can't use the stroke order information, or the stroke direction. Even the speed over the stroke should be able to help you along, especially as you say for cursive characters. Without stroke order it seems almost impossible to decode those in a systematic way.
I suppose this is a general feature of deep learning, it's still not clear how to make good use of all the structural information we possess about a problem, instead a very brute force approach is chosen.
I use Apple's Chinese handwriting keyboard almost every day, living in Taiwan. Unfortunately it only shows 4 possible choices, and often the character I need is not one of those. So I use my own Pingtype keyboard.
Enter a character into the bottom text field, click Search, and see the character decomposition above, and the characters & words containing that character below. Then when you're ready, click Insert to add it to the text field at the top for translation.
I have no idea how to do marketing for this tool, and commenting here is the only way I've had ~1300 views in the last 6 months since I published it.
Characters also have different variants, e.g. the radical on top of 'grass' 草 has variants with 3, 4, or even 6 strokes [1]. There are two 4-stroke variants that look different and have different stroke order. Plus cursive handwriting will often elide and merge strokes to make things flow better.
Full stroke order independence is probably unrealistic (some parts of a character will always be written last), but allowing for some local inversions is probably necessary.
[1] https://en.wikipedia.org/wiki/Stroke_order#Stroke_order_per_...
Deleted comment
People expect more from technology.
So while in the 90's it was amazing that technology could get the correct character if you clearly wrote each character with the correct stroke order, today, people expect to be able to hurriedly scrawl (possibly in cursive) and have the system get the right character. Those are two very different problems.
See for example the images of cursive characters in the linked article. They are very common variations, containing simplifications, but they don't follow anything like the correct stroke order for the character except maybe in broad terms such as writing from left-to-right, or top-to-bottom
I doubt it's anything to do with the fact that people are inconsistent about stroke order - that's true, but they're not that inconsistent and there's no reason a recognition system couldn't account for variations in order just as it does for other factors.
Thanks HN!
For the smaller ICDAR 2013 Chinese Handwriting Recognition Competition (~8,000 characters), CR(1) accuracy went from 94.77% (offline) to 97.39% (online). Human recognition rates were 95.19% (offline) and 96.13% (online).
One of the examples there was a Chinese character being drawn as something like a 2. The character looks nothing like a 2
The best English handwriting recognition software I've found is the old Palm Pilot Graffiti writing system. It's not really developed since the late 90's / 00s. Back in the day, it was blazing fast (and pretty accurate after learning the method) compared to shitty small mechanical keyboards of the era.
After diving into Chinese I realized how deficient the English HR landscape was and how badly under-researched the area is. I think the standard QWERTY keyboard is just too efficient for the Euro zone languages to make the research investment worthwhile. My 2 cents / guess.
There are Graffiti like apps available on IOS. What I've found is that with english keyboard auto-correct, my typing is so much faster without Graffiti using the standard Apple IOS English keyboard. Kind of a disappointment...
Another conjecture, I suspect much of the morbidity in English HR research has something to do with Graffiti patent lock up between what ever remains of Palm, Xerox, and some heretofore unheard of company called Communication Intelligence Corporation.
I stand corrected on everything stated above!
I'll have to borrow a friend's phone to play with this and see how it compares to the old Graffiti.