Thermal images reveal what parts of a screen were tapped | Hacker News Reader

Thermal images reveal what parts of a screen were tapped

theatlantic.com

88 pointsr72172 comments

This was a common mechanic in the splinter cell games, you'd wait for a guard to go through a door with a keypad and you'd put on your thermal vision goggles to find the 4 numbers. I guess that now that thermal cameras are becoming more ubiquitous this has jumped closer to reality. Back then I can't imagine there were thermal cameras that were small and cheap. Nowadays you can get a flir lens for your phone at about $200 - $300 if I recall correctly.

Yep, $250 for the FLIR One [1] or Seek Compact [2].

FLIR has a new version [3] launching Q2 for $200.

[1] http://www.flir.com/flirone/ios-android/

[2] http://www.thermal.com/products/compact/

[3] http://www.flir.com/flirone/products/?id=81752

These things are really cool, we use a standalone one in my office for thermal testing products. It's not just fingers on screens, you can see footprints from someone walking across the floor.

Very cool - any thoughts on the relative strengths and weaknesses of the FLIR vs Seek models?

No personal experience with either. But my quick internet impression: the FLIR's main advantage is that it has a second (higher resolution) visible light camera, which it uses to add an edge detail overlay to the image. The disadvantage is that it has its own battery and has to be recharged.

http://www.phonearena.com/news/Battle-of-the-thermal-imaging...

The problem I had with the Seek model was that it connected to my phone via a Lightning connector, and there was no good way to ensure that connection remained solid. Dropped the thing the same day I got it, and it broke. Haven't tried one since.

It's also hard to use them if you have any kind of a case on your phone.

If I had one that worked via Bluetooth or Wifi, or was built into a phone case, or had a longer cable that I could route around my existing case, then I might give it another try.

Short of that, I'm looking forward to the standalone model.

FLIR's upcoming model looks like you can twist a knob to extend the lightning plug, so it should handle cases better than either of the existing ones.

Re: camera built into phone case, also worth mentioning that the very first version of the FLIR One was a camera integrated into an iPhone 5/5S case. The obvious downside of this is that if you replace your phone with a new model you can't use the camera anymore. This particular design may have a second lease on life if you go to an iPhone SE, but a lot more people moved to the 6/6S/7/7S which won't be compatible.

Standalone versions have been around for years with a really wide array of designs for different purposes (industrial, security, testing, most recently lightweight UAV cameras). For the standalone cameras you pay a bit more because it can't bum off the phone's screen, storage, input, and all that. The TG165 at $350 looks to be the low end unless I'm missing something. No way to browse their site by prices.

http://www.flir.com/instruments/tg165/

Cat Phones (yes, Caterpillar Tractor) offered the first phone with an IR imager. They intended this for industry and construction, where you're sometimes looking for things that are overheating, or heat leaks. It combines the visual and IR images, so you can see the outlines of what you're looking at. Their customer base has a real use case for this feature.

Like other Cat phones, but unlike the USB-port devices, it's not fragile.[2]

[1] http://www.catphones.com/en-us/ [2] https://www.youtube.com/watch?v=mVPku-xItv8

I used a CAT phone for hiking for a while and at first I was quite impressed by the sturdiness (dropped it on rocks multiple meters a couple of times). But there are some design flaws where the connectors are just covered by plastic plugs that leak water after a while and its not waterproof anymore. Would not buy one again.

I'd love to have thermal camera on my smartphone. My application would be that when I come to a bakery, I would like to know which pastry was just baked and is still warm!

LG phones have this optional feature to scramble the digits on the software touchpad so they aren't in the regular order (but a random one). This would be a start I feel; they would need to also actually get a picture of the software keypad instead of just an IR picture your screen however long after

The video they showed seems faked. Why do his other presses and swipes he makes after entering the password not show up in the thermal image? Why would they fake this?

Solution add a feature to warm up the screen to exactly 98 degrees to avoid detection. Learned this from sneakers.

Or present a random 5 digit pin the user needs to enter before entering their personal pin. If the user has touched every button, then you get no information from that.

From the thermal photo in the article, it looks like you get the sequence based on the residual heat intensity. Re-using digits would complicate that a bit, but I don't think pressing half of the buttons randomly before entering the real code eliminates the attack vector.

A scramble-pad is a perfect solution to this issue.

But if you're not going to allow that, but you will allow impractically complicated solutions, then I think you'd be better off asking them to use their pin in a math problem. The CPU could easily choose one that didn't require remainders, to keep it mentally simple.

Sure. Just concatenate a 256-bit nonce to your PIN, calculate the SHA-512 of the concatenation, and then type the last 4 hex digits of the hash. Easy!

In my head, in less than a few seconds, I can take any PIN, calculate AES-encrypt it with a randomly chosen key, calculate the MD5, multiply the result by 65536, and then quickly type the least-significant 4 hex digits. Is that good enough?

For more sensitive areas, scramble pads are usually used. Solves a lot of security issues if you think anyone will go through much effort to get into an area.

For anyone curious like myself: https://www.identiv.com/products/physical-access/utrust-phys...

Wouldn't this burn your fingers?

They meant 98 degrees Fahrenheit - body temperature.

98F (37C) is approx. body temperature, so of course, it wouldn't harm a human body.

The classic units of measurement problem. As an Aerospace engineer, I state the importance of using the proper units in any calculation.

Yeah valid point. I should know better having moved from a metric country (except for pool temperature for some reason) to the US ;p

Another (cheaper?) solution would be to randomise the location of the keys each unlock attempt.

It would cause me a massive problem - I basically only remember my pattern

One of my banks does this for online PIN entry and it's amazing how much it slows you down not being able to do it via muscle memory.

Or move the location of the keys around the screen (but keep the order)

It would also make it impossible to unlock device off-sight (in the pocket) and increase chances of shoulder-surfing.

The proper solution would be let user draw some figure anywhere on the screen then OCR it.

Android -> Settings -> Lock screen -> Scramble pin layout.

If you are talking in general, hmmm yea hard, especially if it's mandatory.

I use a pattern for my unlock, and you don't even need thermal imaging - you can see the smear left on the screen by my finger.

downside: oil from your fingers would cause the touched zones to retain / lose heat at a different rate.

I've seen heated keypads at ATMs.

This is nothing new. I remember an article/paper about the same thing but with pin pads.

Keep in mind, it only works for a very short amount of time as the surface cools down quickly. I don't think it's an attack vector you should worry about.

You can just as easily shoulder-surf the user as he unlocks the phone. It works even better with the "stroke gestures" that is common on android devices - most users don't disable the "draw line" option... it's much easier to spot and remember than a pin pad entry :D

Definitely not new. You can easily do this with off the shelf FLIR handheld devices. Used to work in a data centre and our electrician would use the FLIR to inspect electrical panels for faulty breakers. We were chatting about the FLIR and she was like watch this, put your hand on the table then remove it, you could point the FLIR at the table 10-20 seconds later and still see your hand imprint. Was very cool. I think it cost ~$10k or something though.

As mentioned upthread, FLIR offers smartphone attachments for ~$250 nowadays. Dunno about the quality w/r/t the professional-grade stuff, though.

Sometimes you can unlock people's phones by holding them up to the light and seeing where the grease marks are, if they use the dot pattern unlock or a small pin. Got a few people with this in high school.

I was about to mention this in my post but wanted to keep it short :D. The smudge marks are probably as good of an indicator as the heat signature, considering that they don't degrade with time. It also works better with the pattern-type unlock screen on android (but not so well with the pin-type).

That reminds me, you could build a system that records and highlights the smudges (by using a circular arrangement of LEDs and a fixed camera, or by taking pictures from multiple angles under a single light source). It should make a good weekend project, maybe I'll try it :)

here is a youtube video (https://www.youtube.com/watch?v=8Vc-69M-UWk) from 2014 that demos the technique on pin pads and refers to the following paper https://cseweb.ucsd.edu/~kmowery/papers/thermal.pdf

Without comment on this specific article, there was a video of this doing the rounds last year using a FLIR ONE on a card terminal. I subsequently saw another video debunking it. It only worked if you super cooled the pin pad, there was evidence of that in the original video; it was totally un-workable with a normal room temperature keypad.

I imagine though it's likely highly variable though depending on the thermal conductivity and emissivity of the surface being touched, as well as the actual room temperature and the quality of the thermal sensor.

EDIT: This random video seems to agree with the above, again, no idea how credible. Highly variable on factors. https://www.youtube.com/watch?v=8Vc-69M-UWk

Easy fix. Number pad should come up with numbers in different random order each time.

That'll break for people who use "shapes" on the keypad.

can't fix everything in one go.

but a shape plus a fingerprint ought to keep out anyone except cat burglars and security researchers, at least for the moment.

That solution is even easier, take away the shapes option.

It's not an option. It's what some people implement themselves when coming up with a PIN. Like "square" (4578) or "Z" (1379) or "triangle" (2792), etc.

That being said, I don't think the loss of shape ability would be that big of a deal. Neither do I think this is something that needs to be defended against.

It is an option. Make them use a randomized keypad when setting the pin, and make them enter it three times.

Sorry, my phrasing was confusing. I meant that the pattern input isn't an "option" that phones provide, the way GP comment interpreted it. ("Take away the shapes option") Rather that it is a way some people use keypads.

I didn't intend to say a randomized keypad isn't an option.

I understood what you meant. What I'm saying is you have the chose of whether you will allow shapes to be an option. An always-random key pad makes it physically impossible to use shapes. That is "taking away the shapes option". It's not just the way some people use keypads. You can stop them from doing it on your keypad.

Ah, ok. Well I totally agree.

Imagine numpad in your mind to play shape on to extract sequence

I can't say if this is a serious answer or not.. But either way it does hold something potentially valuable. Show a numpad and have a "connect the numbers" passphrase, just like you draw the shame now. And you connect X numbers that you choose yourself

Next you can just shuffle the numbers each time, making the pattern random whilst still using the shapes.

I guess s/shame/shape/ is in order, took me a while to parse that. :)

Oh yeah, that makes a lot more sense doesn't it! ^^

> Easy fix.

Is there a bigger warning phrase? I don't think so. I have been caught by this as well. When you think it is an easy fix, you haven't really looked at the problem properly yet.

Except when the problem was solved long ago, and you're merely citing that solution.

Sometimes the person asking the question hasn't looked at the problem properly yet.

Why haven't lock screens gone with floating/moving numbers yet.

Or at least a pin pad that randomizes number positions. MMOs have had this 'comfort' feature since the late 90s as a antikeylogger feature.

Most of this risk is mitigated with finger print auth like Touch ID. Though, nothing is perfect.

Fingerprint auth exposes you to court-ordered unlocking though.

not necessarily, if you use several sequential fingerprints in a certain order, and have a hard lockout after X failures. My phone knows 6 of my fingerprints already; shouldn't be hard to get that additional level of "something you know" in addition to "something you are".

I wonder why someone would downvote this without reply? It's a perfectly reasonable suggestion, would be trivial to implement, and would be both a password and a biometric. That's two-factor authentication.

Somebody's a real humbug.

with Apple's rumored removal of a separate home button then multitouch unlock / swipe unlock can become a thing. Then how do you prove the user enabled the feature? With one finger recognition they can just compel you to try each finger and it should work.

Still manufacturers need to add distress recognition. use the wrong finger and it dumps all of them requiring pin only

Biometrics aren't cryptographically secure. You can't deterministically derive a key from biometric data.

Moreover, you can't rotate your keys.

Actually you can: http://web.cs.ucla.edu/~rafail/PUBLIC/89.pdf

Cool!

From my cursory reading, it doesn't seem like you can rotate keys, though.

Good thing I always carry my phone in the back of my jeans. Hot pocket, FTW.

Isn't this used for skimming ATMs?

What's new about this?

useless attack since many phones are already randomizing number pad each time is displayed

no shit

I recommend that everyone use an extended-length PIN for your phone. Both Android and iPhone support it. Mine is 12 digits; a bit of extra time, but vastly more difficult to brute-force or shoulder-surf.

Why digits and not a real password? I have found that typing 10 letters is not too inconvenient on a phone.

The size of the alphanumeric keyboard is a real problem for me when trying to enter a password to open my device.

Sure, I can do it, but it really slows me down, and makes the value of the password a lot less to me.

I'd rather use a longer and more complex PIN on a much larger keyboard. Preferably one that re-uses at least some of the numbers, so they might have an idea of how long the PIN is, but they might have a harder time figuring out what the correct order of the numbers is.

At least, that's my current view. That might change tomorrow. ;)

Doesn't this depend on how worried you are about brute force and shoulder surfing? I am pretty sure in 99.9% of the time that someone is trying to access my phone for nefarious purposes is going to be someone who stole my phone when I left it somewhere. I have no reason to worry about an advanced attacker.

In this case, why would I make the trade off of convenience for security? I have to do 3 times the work to try to defend against an insanely rare attack.

Reply on news.ycombinator.com