Hacker News Reader

Comment by ladberg | Hacker News Reader

I agree the keyboard will remain as the best text input device but I'm pretty confident that eye tracking will be categorically better than a mouse/trackpad. I'll expect that eye tracking solutions for normal monitors will also become more mainstream once people get used to it in the Vision Pro and want to use it elsewhere.

View on HN

Eye tracking will probably lose the productivity metric, though. When you remove the ability to look at one thing and interact with another, you get farther from the way that humans tend to interact with their immediate physical environment.

I can cut vegetables without looking at them. I can use that dynamic to offset the planning and acting phases of my thought process. Falling short of that efficiency will feel limiting.

I don't click without looking at my mouse pointer. I think it's possible eye tracking is just better than mouse/trackpad.

I often am looking at another element before interacting with it with my mouse. With eye tracking enabled menus hovering over an element immediately because I'm looking at it becomes an annoyance.

Yea, I don't think a 1-to-1 translation of mouse movements with eye tracking is the correct answer here. I do think it's probable that eye tracking + {X} can be a 1-to-1 translation for clicking for 80%+ clicks.

Maybe X is a button on the keyboard. Maybe X is a gesture.

I do, when gaming.

I can think of some Portal puzzles in particular where timing is important, and you need to hold your aim but wait to click until something happens somewhere else on the screen (so the place you're clicking is not the same as the place you're looking).

I think the same thing applies to e.g. recording network activity in Chrome dev tools. My eyes are on the page to see when the thing I'm interested in finishes loading; my mouse cursor is on the button to stop recording.

It's not a super common pattern, but probably common enough that it would be annoying not to be able to do it.

Yea, I agree with this for gaming. Eye tracking as a gaming interface probably requires rethinking a lot about games. In VR for example the movespeed is much slower. In normal video games the character movement is constantly super-human speed, and this is jarring while in VR.

I am speaking mostly about the desktop interactions. In your Chrome Dev situation, I would look at the cursor before clicking on the stop recording button. I think I might be able to trust the MBP trackpad to do a primed click without looking at the cursor, but I wouldn't trust a traditional desktop mouse to have stayed steady enough.

Interesting, I just tried to use my pointer with my peripheral vision, and while possible, it was indeed more difficult and harder to focus on. Something I hadn't tried before.

Eye tracking has been $100 to own for many years now through a certain company. Why hasn't it taken off?

The mouse is the superior input device. When people who actually need a better input device get one, they get more advanced mice:

https://3dconnexion.com/us/spacemouse/

I'm not sure what the SOTA is currently but the Vision Pro has much better eye tracking that other systems I've used in the past. Idk if the sensors are actually better or if it's just the native integration into the OS and apps, but it feels more seamless than a mouse and I've never "misclicked" on the wrong thing with my eyes. It kinda feels like if the mouse cursor instantly jumped to the right place every time with no travel time or physical movements required.

I've used a 3D mouse for CAD but am not sure where else it would be helpful?

Tobii https://gaming.tobii.com/product/eye-tracker-5/ is standard in consumer eye tracking, they even have integration with some popular video games! There's just no value in it. I tried an older version in a store, and it was basically perfect, tracking my eye's movements exactly, with zero delay.

Nobody wants it for day to day computer interaction. Most people using eye tracking for computer interaction are disabled, because it's a terrible experience.

I primarily use i3wm with 4-40 terminal windows + firefox open. I'd LOVE it if the terminal that I'm looking at was the `active` one. I can't count the number of times I'd lookup some error code, type in more commands, hit enter; only to discover that I had been typing away in the wrong window.

This will be no problem at all in the sdk. Hell I saw someone do this as some point on the mac using the macbook camera popping windows to the front based on eye-dwell... I think they started with cursor control.. essentially eye-mouse.

Oh wait some version of that is built into accessibility on mac already (eye tracking mouse): https://support.apple.com/lv-lv/guide/mac-help/mchl437b47b0/...

As we use an iPhone so close to our face, I would love to see an eye tracking system work for them. But maybe it really need that incredible close proximity, two cameras per eye, and IR illumination in the Vision Pro to work.

I definitely want to see a monitor with Vision Pro-style eye tracking and hand tracking. I'd love for eyesight and subtle gestures to fully replace pointer devices.

Voice input is going to replace the keyboard shortly.

Sure, just right after Linux takes over desktop.

How do I click though? Blink? What about a right-click?

Asking me to "blink twice" or anything like that is going to make my eyes lose focus on the screen/content

The pinch gesture on Vision Pro is pretty smooth and easy, about as much physical exertion as a mouse click. Right click is harder, I'd expect most app developers to do something similar to iOS and treat long clicks as right clicks but that's pure speculation.

I'd love to have an eye tracking setup though on a normal desktop computer where I could devote a keyboard key to clicking and I'd never have my hands leave the keyboard.

On the Vision Pro, tap your index finger and thumb together.

Maybe they could add other features from just your face though for other platforms, wiggling nose, raise eyebrows, stick out tong, blow a raspberry?

That or they add voice commands, "open", "select" and such.

I suppose ultimately in spatial computing with voice interface and eye tracking the concept of a "click" may die.

There's also apparently other gestures and voice commands that can be selected in the settings, though I haven't seen any real detail on it other than "they're there".

Reply on news.ycombinator.com