Gesture Recognition with Line Integrals | Hacker News Reader

Gesture Recognition with Line Integrals

justinmeiners.github.io

51 pointsjustinmeiners16 comments

Not sure what is intended, but it seems to have some bugs?

https://assets.opentoken.com/sha256/N1ShZ7kcVmH_-auZEGaGOS_b...

Ah nevermind, it's matching the curve against template fields so it makes sense that it would show the best match even when the match is poor.

> best match even when the match is poor.

Yeah, a real application could have a score threshold.

I created a method similar to this one for curve extrapolation based on sample curves. I took the second derivative of each point on the training data, added it's contributions to a vector field.

To extrapolate, you can take 'position' and 'velocity' of live data and integrate over the vector field to produce an extrapolation. I enjoyed the project and it worked fairly well. I think there's plenty of room for extension around this method.

It was ridiculously hard to get 4 to work, is this a limitation of the algorithm or just something with the implementation?

How do you draw your 4? The one I used is the "one stroke" 4 where you draw it from left, top, and then down.

I draw mine like a capital H minus the lower left segment and I gave up trying to get it to recognize it. It kept coming up 7 or 9

I quickly realized variations of:

    │  │
    └--┤
       │

Weren't going to work so I switched to the style in the font:

     ╱│
    ∠--┼
       │

And it really liked to give 7, 6, or 2 if I went in the left,upRight,down pattern.

I think I found out the trick though: if you make a sharp 4 like the font you get random results but if you just make smooth loop of it (almost looks like a tilted roller coaster) it gets it every time.

Even went I deleted the preset glyph patterns and spent 5 minutes re-training it for 1-9 my training pattern for '4' still didn't work. I tried drawing it 2 different ways. It's neat that you can train it so quickly with the hold-shift feature though.

Cool but how is the math used to recognize gestures?

It appears to be a dot product of a vector field and the curve vector (i.e. a line integral).

I think this approach can't handle certain cases where a path is retraced in opposite directions, like with the handwritten letter 't' or 'i'. Perhaps curvature integrals could be added as well to improve it.

Overall seems to work quite well!

> can't handle certain cases where a path is retraced in opposite directions

It does ok if there is only a small part that retraces, but you're exactly right.

I think it works quite poorly: https://imgur.com/a/oDpVkZ6

Looks like you draw your digits differently. It's "trained" to my style. Try making your own glyphs and see how it does.

Perhaps, but how does that 9 I drew ever map to a 6?

I don't think so. I draw my 9s by drawing the circle, and then the line down. Looks like you go the other way. This is all about direction and gesture as opposed to OCR.

This animation might help:

https://en.wikipedia.org/wiki/Line_integral#/media/File:Line...

Reply on news.ycombinator.com