Hacker News Reader

Comment by wongarsu | Hacker News Reader

So for context I looked at the papers behind the scenarios.

- The heart rate sensing is done on a smart watch, not a phone, and needs data from the actual heart rate sensor every couple of days [1].

- The breathing rate is determined from a phone put on the breast or the abdomen [2]. Not really a threat vector in that form.

- The audio stuff is incredibly impressive [3], but it doesn't look like they can reconstruct text with meaningful reliability, it's more about identifying the person or at least the gender of the person on the other end of the line.

The location and activity detection scenarios seem the most credible to me, but the for targeted attacks the audio reconstruction might also work. The other two don't really seem credible to me yet, but good to be aware of them.

1: https://arxiv.org/pdf/1807.04667.pdf

2: http://www.ijetch.org/vol8/900-M302.pdf

3: https://arxiv.org/pdf/1907.05972.pdf

View on HN

One of my pet peeves is that these articles never report the a priori distribution of labels that are being predicted. Guessing whether you're at sitting vs walking with 90% accuracy is not very useful if most people are usually sitting. Actually, the article here doesn't even report the accuracy: neither the stdev error for scalar predictions, or the precision/recall and number of classes predicted for ordinal data. Actual results would go a long way to making the article meaningful.

There's a growing number of these "computers can now predict..." articles, like "one email from you can tell your mood" or "your choice of snap filter reveals your age." Basically, in practice anything can predict anything, usually slightly better than random, so there's an infinite number of articles to write that sound shocking. If I had a bit of training data, I can easily write a script to predict everyone here's salary based on their comments using bag-of-words, and I bet I can do better than random guessing.

For what it's worth I did a fair bit of research on using accelerometers for heart rate and respiratory rate determination (https://ieeexplore.ieee.org/abstract/document/5504743) and while placement on the abdomen/chest is preferable, if the subject has a high excursion and low adipose tissue I was in some instances able to get a good signal from a device worn in a trouser or jacket pocket (especially when sitting). Never published that because it wasn't useful in a medical setting but could be of interest if you're recording it opportunistically.

Using the accelerometer/gyro data, you can also guess the password from the device with quite good accuracy(it depend) if you type in your hand and not on a flat surface.

No you can't - apps can't read Core Motion data when your screen is locked, or they're in the background.

I don't have an iPhone, but does it have overlays for things like password managers?

Android definitely does, and I assume the same research around guessing text input would apply.

It does, but I unlock it with FaceID rather than a password. Cameras recording me typing my bitwarden master password are a bigger security concern for my password security already.

Wish there was an option to disable the magnified, per-key, onscreen keyboard bubbles when entering a password. Provide an option to show the password on entry or not, same as desktop input fields for passwords. But don't magnify each character and invert the screen to make it easier for camera imaging.

There is, at least on iPhones. But it is for all keyboard input, not just passwords. Go to Settings -> General -> Keyboard, toggle Character Preview to off.

Thanks, that's helpful, although the iOS lock screen appears to be excluded.

You may create App such as a (kind of)"navigation" app. But I'm not sure about collection data while screen is locked and password is required. Need to play with it.

I would guess the vast majority of people who need to set a PIN for the CVS app or their banking app or whatever use the same PIN to unlock their phone.

If they’re creating a PIN in the app, the developer could just store that in plain text more easily than trying to derive it from the accelerometer. I think you’re right that most people probably use the same PIN for their phone and apps, but then the app maker has easy access to the PIN because the user gave it to them

Is there an application on iOS that can listen for the accelerometer/gyro data on the background?

Health definitely does (and likely needs to for much of its functionality), but it’s first party so you’ve pretty much already trusted the vendor with that data.

Future Interfaces Group at CMU has done a lot of interesting research where they can infer a large amount of enviromental context just from accelerometer/gyroscope alone. http://www.figlab.com/

Some examples:

https://www.youtube.com/watch?v=a-ImMjOrbbI

https://www.youtube.com/watch?v=gCA-0cPS1eM

They also have a bunch of research using other sensors in the phone as an input.

Wow, some of their research is pretty insane. I'll have to play around with that at some point.

It's really insane. Chris Harrison is one of the best minds in HCI in my opinion. Taking his class at CMU was a highlight of my experience there.

Reply on news.ycombinator.com