reynaldi

Glad you enjoyed reading it! I just checked Project Gameface demo [1], and really cool that it is accurate enough for drawing text, I wonder what it is tracking. Are you still using it?

[1] https://blog.google/technology/ai/google-project-gameface/

Interesting, never heard of the IIR filter before, will keep in mind as one of the options if I ever worked with removing noise again, thanks for sharing!

Wow interesting, reminded me of that Meta Orion wristband, I wonder if that is the goal.

I'm glad it is up to the HN standard :) No, I don't currently use it, I am back on mouse and touchpad, but I can definitely see what you mean by remote movie control. I would love to control my movie projector with my hand.

I've been thinking on and off on how to improve the forward facing mode. Since having the hand straight ahead of the camera is messing with the readings, I think the MediaPipe is trained on seeing the hand from above or below (and maybe sides) but not straight ahead.

Ideally, the camera should be like kind of above the hand (pointing downwards) to get the best results. But in the current version of downward facing mode, the way to move the cursor is actually by moving the hand around (x and y position of the hand translates to x and y of the cursor). If the camera FOV is very big (capturing from far away), then you would have to move your hand very far in order to move the cursor, which is probably not ideal.

I later found the idea of improvement for this when playing around with a smart TV, where the remote is controlling a cursor. We do that by tilting the remote like up and down or left and right, I think it uses gyroscope or accelerometer (idk which is which). I wish I have a video of it to show it better, but I don't. I think it is possible to apply the same concept here to the hand tracking, so we use the tilt of the hand for controlling the cursor. This way, we don't have to rely on the hand position captured by the camera. Plus, this will work if the camera is far away, since it is only detecting the hand tilt. Still thinking about this.

Anyway, I'm glad you find the article interesting!

author here, sorry you have to see my janky JavaScript solution XD but one good thing of going with Tauri is that developing the UI is pretty easy, since it's basically just some web pages, but with access to the system, through the JS <-> Rust communication.

also, rewriting neural network from PyTorch to Java sounds like a big task, I wonder if people are doing ML in Java

Awesome, I was just about to look for something like this and it showed up on HN!