Keylogging on IPhone and Android Using Gyroscope Data and Machine Learning

I was a bit disappointed by the lack of a real scenario implementation. While the data suggests it might work, I wanted to see how precise it would be in a simple implementation like the author described towards the end, where a simple game of tapping trains the NN and then uses the information to try and capture a typed password.

If I got that close to making something as cool as this, I would not stop experimenting until I found out whether it worked or not.

That is, unless I tested it, found that the imprecisions made it impossible to obtain reliable data, and decided to not write about it to not discredit all the work I had done.

I really hope someone takes the idea and tries it out to assess how viable it is, then I can be paranoid about 2 more sensors spying on me.

Very cool!

Also this is mindblowing, logging keystrokes from a standard keyboard using wifi.

"we show for the first time that WiFi signals can also be exploited to recognize keystrokes. The intuition is that while typing a certain key, the hands and fingers of a user move in a unique formation and direction and thus generate a unique pattern in the time-series of Channel State Information (CSI) values"

https://www.sigmobile.org/mobicom/2015/papers/p90-aliA.pdf

Edit: I wonder as an aside how precisely you could potentially use the accelerometer etc. for dead-reckoning.

Attacks like these can be easily dwarfed by employing a nonstandard virtual keyboard which displays a randomly permutated key layout during PIN entry. Some online banking web apps did this already ten years ago as a defense against "mouseloggers". Using a good RNG is advised though.

Wouldn't the model be different for each user? and different for different typing scenarios? Like typing one handed vs two, sitting on a bus, in a car, while walking, versus just sitting on the couch.

It seems like it could be possible, but it would require a lot of training. Get it into a popular note-taking app to record keystroke + accelerometer data.

Also a simple fix for iOS as a platform would just be blocking out or filtering motion data when the secure keyboard is showing. I assume a similar thing could be done on android.

TL;DR it's possible to detect where you tapped on your phone screen somewhat reliably. Theoretically, someone could develop an app to associate screen location taps to keys and could pick out a password. Authors suggest phone manufacturers should limit access to accelerometers to prevent the attack.

This is extremely contrived and would take a shitload of skilled work to get right. It's way easier to make a phishing page coupled with social engineering to get what you want.

Neat, though i'd call it key-estimation rather then logging, as i consider logging a very accurate, 1:1 log of what actually happened, this could have some errors in it, not making it less 'dangerous', lack of better word.

But for now i'll have to randomly rotate my device at each input... And or get one of those keyboards that shuffles the letters around the keyboard at each input.

Come to think of it, I can't remember the last time I typed a password in an app, after initial setup.

Quite surprising iOS allows that kind of sample rate for a background app.