Show HN: A tiny JS library for real-time localization of eye pupils
tehnokv.com
tehnokv.com
Using decision trees for real-time localization of eye pupils
https://tehnokv.com/posts/puploc-with-trees/
It's actually quite fascinating. I'm too paranoid to enable webcam for any random site, so will be trying it out locally.
For face detection and determining the locations of the eyes, the demo uses picojs, described in another post:
pico.js: a face-detection library in 200 lines of JavaScript
1 - https://www.ag.state.mn.us/Consumer/Publications/GrandParent...
The collector if the data doesn’t need to be nefarious, they only need to be sufficiently careless.
I suppose I used the word "paranoid" to imply that it was not entirely logical/reasonable to think so.
Thinking of it, I acquired this attitude of being overly careful since Facebook, face recognition, machine learning, pervasive privacy invasions became the norm on the web. Who knows, maybe there's a way to combine random webcam snippets with other tracking technology to gather even more data about me.
Yes, probably the height of laziness to avoid pressing Alt-Tab but its interesting to me.
Edit: Its possible that i am mistaken and what I would need for that is more gaze-tracking than pupil-tracking.
> Television viewing tends to be a passive experience for a viewer[...] To increase interactive viewing and encourage a user to watch one or more particular items of video content, awards and achievements may be tied to those items of video content. [...] Producers, distributors, and advertisers of the video content may set viewing goals and award a viewer who has reached the goals.[...]
> Additionally, the viewing behavior may include an action performable by the viewer and detectable by one or more sensors, such as a depth camera.[...]
> If the viewing goal has been met, an award may be granted to the viewer. An award may be a virtual award. Such as an addition to a viewer score or an update to an avatar associated with the viewer
Basically, if the phone was playing video and was no longer pointed to your face, it would pause the video. I could be wrong on the specifics here, but that seems close to what the feature advertised was.
This demo seems to do pretty well -- e.g. it handles covering one eye or arbitrary parts of your face, but has some trouble at oblique angles (like your face in profile or near-profile).
But here's an odd thing I noticed:
- Take a pair of over-the-ear headphones ("cans").
- Rotate them 90 degrees so that one earpiece is on your forehead (or above it) and the other is at the back of your head.
This seems to completely stump the algorithm, even when your eyes are plainly visible and looking straight into the camera. With my headphones turned sideways it never once identified my pupils (or my face) accurately.
Possibly a hat would do something similar (I don't have one handy) but a largish dark circle your forehead seems to confuse it.
Very neat for a 200 LoC browser-based demo.
My glasses are closer to the "invsible frames" side of the spectrum than the Harry Caray / Elvis Costello style, so maybe that matters.
I guess maybe I should just review the code but I was trying to figure out if it uses other facial features to "anchor" the eyes or just the (fairly recognizable) features of human eyes themselves. Your glasses experience makes me guess it is pretty eye dependent -- it does have more trouble with one eye covered for example, whereas if it was anchoring off some combination of ear/nose/mouth/chin it should be easy enough to ID the eye from half the face, let alone with just an eyepatch-sized cover.
That's a false positive. This uses quite simple algos, decisition trees, which have more false outcomes (both positive and negative) on this pupil detection task that state of the art models, which would be deep convolutional networks. The decision trees are much less computationally expensive though - running a modern deep CNN at 30 fps needs a dedicated GPU. The decision trees will run just fine on a CPU in a browser!
I've also been looking for usable gaze tracking, to play with combined gaze (fast but low resolution) and head (high resolution but slow) pointing. Perhaps lploc can help with that.
So thank you for your work.
[1] PoseNet webcam demo: https://storage.googleapis.com/tfjs-models/demos/posenet/cam...
No doubt it "just works" for some people. For others, not so much.
If you add this CSS to your site, you can increase the readability without sacrificing the aesthetic.
body { max-width: 65em; margin: 10px auto; }
...and then?
I'm guessing your browser is blocking the webcam, or maybe its not configured correctly to work with your browser. did you get a popup requesting permission from your browser? sometimes its hidden at the edge of the URL bar behind an icon.
I would love an interaction method like this.. I can already think of some super fun UIs to build around it...
Edit: Nevermind, it requires access to canvas data which for me is blocked by default.
Seriously though I think if it's all JS based you should be good to use incognito and offline to gate off the code from uploading.
Presumably you could close both eyes but I can't test that for obvious reasons.
So this specific implementation might not work for that.
Really?
I couldn't find what I was actually looking for, where Weizenbaum describes how vision or reasoning experiments might be made with benign or even cute objects, but for rather not so benign ends. I found this instead, which I think is even better put.
> Other people say, and I think this is a widely used rationalization, that fundamentally the tools we work on are "mere" tools; This means that whether they get use for good or evil depends on the person who ultimately buys them and so on.
> There's nothing bad about working in computer vision, for example. Computer vision may very well some day be used to heal people who would otherwise die. Of course, it could also be used to guide missiles, cruise missiles for example, to their destination, and all that. You see, tthe technology itself is neutral and value-free and it just depends how one uses it. And besides -- consistent with that -- we can't know, we scientists cannot know how it is going to be used. So therefore we have no responsibility.
> Well, that is false. It is true that a computer, for example, can be used for good or evil. It is true that a helicopter can be used as a gunship and it can also be used to rescue people from a mountain pass. And if the question arises of how a specific device is going to be used, in what I call an abstract ideal society, then one might very well say one cannot know.
> But we live in a concrete society, [and] with concrete social and historical circumstances and political realities in this society, it is perfectly obvious that when something like a computer is invented, then it is going to be adopted will be for military purposes. It follows from the concrete realities in which we live, it does not follow from pure logic. But we're not living in an abstract society, we're living in the society in which we in fact live.
> If you look at the enormous fruits of human genius that mankind has developed in the last 50 years, atomic energy and rocketry and flying to the moon and coherent light, and it goes on and on and on -- and then it turns out that every one of these triumphs is used primarily in military terms. So it is not reasonable for a scientist or technologist to insist that he or she does not know -- or cannot know -- how it is going to be used.
-- Joseph Weizenbaum, http://tech.mit.edu/V105/N16/weisen.16n.html
I read the GP completely agreeing with your point, but this comment is such a clear reminder that we have no hope of knowing what change will come from a technology that I had to change my mind.
> I read the GP completely agreeing with your point, but this comment is such a clear reminder that we have no hope of knowing what change will come from a technology that I had to change my mind.
Okay, but why? Because a military application doesn't come or come to my mind when I "think about computers and lasers"? I'm not following.