Real-time tracking of human eyes using a webcam | Hacker News Reader

Real-time tracking of human eyes using a webcam

codeproject.com

34 pointssoundsop11 comments

There is a Swedish company called Tobii that does something similar yet a lot more advanced, including lasers and what not to track eye-movements.

One of their ideas is that people with physical handicaps could use their eyes instead of mouse/trackpad to point and click at stuff, but if this could be done with a webcam that's even more better. If you could click on you IPhone only using your eyes, wouldn't that be cool?

However, it's quite improbable you'll get an interesting resolution for your movements with a webcam, compared with lasers or other methods. Image-based eye tracking needs to be real-time (so probably inherently low resolution these days), tracks objects that are an important part of the captured image (so you get like a much lower resolution mouse, because of the size of your eyes or pupils compared to the webcam resolution), and needs to compensate for errors so it needs a lot of redundancy in data to generate output (so decreasing the resolution even more).

Anyone working in computer vision can tell me if I'm right or wrong? I've been working in the area in some small projects, but they were extremely related to this eye-tracking stuff.

I know from using the tobii that eye-tracking has one big problem: Your eyes never stay still. Whenever you're focusing on something, you are making mini-movements (called micro saccades IIRC) around the focus point. When you move to the next point, you're eyes 'jump' there and do the mini-movement again. When you read a line of text, it feels as a fluid movement, but it actually is a series of jumps over a couple of words at a time.

Interpreting eye tracking is mostly defining what is the cut-off point between movement during focus and actual movement. It seems that with a web-cam, this choice is already made for you, due to it's low resulution :). It's been a year or two since I had to deal with it though, so my memory is a bit hazy. I can post links to some good articles on eye tracking if anyone is interested (just leave a reply).

Finally, I'd like to remark that many people put their mouse where their focus is. Mouse tracking is a quite effective (and cheap, and hardware-less) means of focus logging. As is simply recording clicks.

You're dead on. The purpose for the micro-saccades is to keep the image on the retina changing. Photoreceptors on the retina do not respond to the intensity of light, but rather the rate of change of intensity. A combination of micro-saccades and blinking lets your visual system gather as much information as it can, and visual memory fills in the gaps.

The illusion of a complete image is really one of the most remarkable things about the visual system.

A webcam, eh? I always keep track of my human eyes using egg crates. They're just the right size, and you can label the outside with sharpie...

Somebody needs to commercialize this, stat. I would love to get visual heat map of my web site - which portions of the text user looks at?

Few things I can think of..

1. Usability tests (including heat maps and replays for manual observations)

2. Measuring effectiveness of display ads (web, TV etc) more concretely.

3. Accurate Strabismus (Squint) recognition and measurement.

http://www.seeingmachines.com/faceAPI.html

Now we just need to hack together a Johnny Lee style 3D demo like the one in the faceAPI video...

Heat maps could also be utilized to provide a computer system a probabilistic notion of the "current object" that the user is referring to. If a computer system could respond to where the user is looking, this could augment verbal input greatly. The computer might actually stand a chance of knowing what the user means when she says, "I want you to look up that."

I can't seem to get this thing to figure out where I'm actually looking. Isn't that the point of "eye-tracking"? It is pretty good at figuring out where my eyes are located on my face, but that isn't very valuable information.

Has anyone else had any success?

Is not it too much stressful to use eyes as an input device? I don't think there general uses for people that can use a mouse. I think that the next revolution in input devices can be a keyboard that is actually like the iphone a display + touchscreen so that the keyboard will drastically change in order to be application-specific.

Reply on news.ycombinator.com