Show HN: An Augmented Reality Hand Puppet App

medium.com

11 pointshwoolery5 comments

Awesome idea - tracking hand movements can't be easy! I like the dino puppet.

cool work! i was also working in a similar game/computer vision space its very fun

and also

> hand gesture recognition is far more complicated than facial feature detection

the good news is we may not be that far off anymore https://news.ycombinator.com/item?id=14263678

thanks! I saw that page as well a while ago on HN, although I'm not sure how feasible it is for such a network to run smoothly on mobile.

I'm no expert in anything deep learning, but I think getting it to work on mobile is just a matter of importing your trained file like the dlib facial landmark detector. which definitely runs in real time - albeit maybe not at 60 FPS.

yup, I actually use the dlib facial detector in this app for a ventriloquist type feature (not shown in video). The facial landmarks detector assume you've framed an ROI for a face already. Locating faces in an image is trivial... locating hands in an image and their orientation, not so much : ) That's not to say a CNN couldn't be trained to do it, it's just going to take some work!

Reply on news.ycombinator.com