Wi-Fi signals enable gesture recognition throughout entire home | Hacker News Reader

Wi-Fi signals enable gesture recognition throughout entire home

washington.edu

432 pointsrisent77 comments

>A standard Wi-Fi router could be adapted to function as a receiver.

The article and the embedded video do not make it entirely clear but if this technology could be deployed in existing Wi-Fi routers with relative ease [1] then it could have far-going implications for both privacy and the physical security of anywhere where there's Wi-Fi. Once it's easy enough to deploy on a hacked device that a hired script kiddie can do it even simple burglars could take an interest in residential routers to know when nobody's home.

I wonder if this will affect the market for embedded Linux security consulting in a major way.

[1] I.e., without modifying the hardware and with no need for manual calibration.

Check out another video from the article

http://www.youtube.com/watch?v=wK_u8-UQmOs

Send ultra sound from the laptop speaker, pickup with mic, detects hand guestures.

From the paper [1]

The WiSee proof-of-concept is implemented in GNURadio using the USRP-N210 hardware.

Each USRP is equipped with a XCVR2450 daughterboard, and communicates on a 10 MHz channel at 5 GHz. Since USRPN210 boards cannot support multiple daughterboards, we built a MIMO receiver by combining multiple USRP-N210s using an external clock. In our evaluation, we use MIMO receivers that have up to ﬁve antennas. We use single antenna USRP-N210s as transmitters.

So basically, they are using 6+ USRPs at $1400+ each to do these experiments. However, since each USRP is communicating on a 10MHz channel at 5GHz, someone with between two and five MIMO 802.11n Wi-Fi devices could theoretically do the same.

[1] http://wisee.cs.washington.edu/wisee_paper.pdf

I think you'd still have to hack the routers to provide the external clock, as the whole scheme depends on accurate sensor fusion between multiple antennas.

Not only you have to hack the routers, once inside the router you'll have to flash a custom wireless firmware, or "radio" firmware. And also the Chipset must be flexible enough to allow all the iFFT shenanigans. I think it's a stretch to say you could do this with a common wifi device, maybe with a very specific wifi chipset you could, and maybe you could make it work with bluetooth or wireless phone chipsets aswell.

That said, it's an excellent paper about a soft-based radar.

IIRC, I think there are some weird FCC rules about external synchronization of radio transmitters. Using the clock would be ok, but synchronizing using out-of-band signals is a problem for some reason. (This is hazy, but I remember working around issues synchronizing UHF (915MHz) RFID readers).

Honest question, how can you have a 10MHz channel at 5GHz?

"5GHz" really means 4.915GHz-5.825GHz, the frequencies in which 802.11n is allowed to operate. You could split that up into many 10MHz wide "channels" of non-overlapping frequencies.

Even with enough MIMO 802.11n devices it's still practically impossible. They essentially need access to the raw signals to do all their calculations, but this isn't provided by drivers/firmware/devices.

If you're really lucky you have a device with open source firmware [1]. However even that firmware can only interface with the PHY layer by writing to registers to change the configuration of the device. Essentially the modulation of the signals is done in hardware, and you only control MAC aspects of it (things like disabling carrier sense is possible, changing backoff behavior, inter-frame wait timings, etc). But you can't access the real signal, it's a hardware limitation, so this not possible using existing devices.

[1] https://github.com/qca/open-ath9k-htc-firmware

It's already been done:

http://www.extremetech.com/extreme/133936-using-wifi-to-see-...

Determining when nobody is home is already not that hard. Just ping the devices registered with DHCP, over a few days. Should be fairly easy to identify which devices are phones and alert you when none of the phones are responding.

Just need to ensure a proper counter measure is (1) available and (2) legal.

This sounds a lot like the mobile phone sonar that Batman used in the Dark Knight which seemed utterly ridiculous to me and many others at the time:

http://sfblunders.wordpress.com/2008/08/07/dark-knight-sonar...

Ok DD-WRT folks, get to it :-) More seriously though, this is a pretty cool thing, using your motion which is blocking wireless signals to detect motion. In hindsight its kind of obvious since people are 'giant waterbags of attenuation' as my RF buddy calls them. This also means you can probably add something of a burglar alarm/motion sensor system to these things, and if you were careful you might be able to see the movement of people inside a house by looking at the WiFi signal leaving the house. Certainly if you're carrying around and active phone/tablet that would be trivial to do from outside (and by trivial I mean you would have lots of signal to work with and the math would be straight-forward, not that it would be simple)

We can track people, cool! And probably kids too! What about dogs and cats? Rats, mice, and cockroaches?

What's the range on this technology, and what happens if we move the router around with us? Maybe we take it to a new building and plug it in there? Or maybe we can use the wifi modules on phones to do this also? It sounds like a great scanning device. And how long before we can also see huge chunks of metal or strong magnets with this technology, based on how radio waves interact with them?

6cm is the wavelength at 5GHz and most sensing devices have issues with sub-wavelength objects. If you're paranoid you can hang a bunch of strips of aluminum foil that are 3cm long all around your house, which could confuse the sensor. (see http://en.wikipedia.org/wiki/Chaff_(countermeasure))

Work is underway on 60Ghz Wifi, which has wavelengths of only a few millimeters.

http://www.pcworld.com/article/2030041/meet-60ghz-wi-fi-the-...

In a few years we could have tri-band routers covering 2.4Ghz, 5Ghz and 60Ghz. Correlating the info from all these frequencies should provide a much better picture.

Oh the possibilities of this - may be with enough data - you can just wander around a suburb and figure out which houses are currently occupied and which ones are not. - improve aged care by monitoring walking patterns - combine it with some more ingenious technology for localised sound -- you can beam music as I walk around the house (or block out sounds)? - a bit more sensitivity and we can even probably pick up a person while they are having or just about to have a heart attack?

I am sure the privacy folk out there are having multiple heart palpitations.

All I want to say is "bring it on"

It seems easy enough to reduce leakage or fuzz the signal.

I don't have the background to try to pull the limits up out of the white paper, but I think the current implementation is more or less looking for movement in 1 dimension (towards and away from the receiver). So plenty of time to decide whether radio waves are revealing anything that sight and sound are not.

And maybe predict domestic violence before it happens.

I'm not sure if I qualify as one of the "privacy folk" or not, but I would say "As long as the police cannot compel you to install the thing and switch it on, or have an AI system issue search warrants in seconds based on what the machine reports." Unfortunately I think we will have both of those things 20 years from now.

An AI system issuing warrants defeats the entire purpose of warrants. The entire purpose is to put human oversight into the system.

Whether you consider this a fatal objection to the idea or a problem to be solved is an exercise for the reader. (The future is weird.)

There is no need to force anybody to install and turn on anything - they could just passively monitor the signal of your wireless router, our that of your neighbor or of your cellphone. Or they could actively send such a signal and analyze it if they don't care about being detected. In the end it is some kind of (passive) radar and you will probably have to actively jam or block it in order to prevent someone from monitoring you.

Look out for lots of <img/> tags pointing to http://192.168.1.100/epic_firmware_fail.cgi?sploit=PAYLOAD!

> Neighbor: Why do you always wave your hands around?

> Me: It's the preamble to the spell where I invoke the deamons

> Neighbor: Why the stick?

> Me: Improves gain

I've often wondered is to what extent existing RF noise as "illumination" could be leveraged to effect some kind of passively acquired representation of objects. Resolution would of course be limited by wavelength but improved if the antenna array was physically moving in a describable manner. We had useful SAR during the cold war and orders of magnitude more compute power today. If you know of more efforts along these lines from hacker to research level, please post a link.

Can this differentiate between multiple humans gesturing at the same time (intentional or otherwise)?

What about pets moving around or turning the shower on?

Your pet turns the shower on? That sounds like a real problem

I think he refers to having a system that, using this approach, turns on his shower when he enters it. If this cannot distinguish humans from cats, it would turn on the shower when a cat entered it.

I think that's a bad example. The typical cat would soon learn not to enter the shower.

> I think that's a bad example. The typical cat would soon learn not to enter the shower.

My black lab, however, would enter the shower every time I forgot to close the door.

I'd be more concerned about when your kids figure out the gestures control things.

If the system has enough lag, clever cats might dart in and out before the shower sprayed for a brief bit, so as to get some fresh water (which they like more).

One of my cats has actually done this a couple of times. He walks along the ledge around the bathtub and squeezes past the shower handle, forcing it away from the wall and turning on the water. It's a problem that seems to have solved itself, because it scares the shit out of him and he gets wet.

I think this is where AI comes in -- it should be possible for AI to discern meaningful/non-meaningful movements (based on size, speed, previous state, etc.)

" If a person wants to use the WiSee, she would perform a specific repetition gesture sequence to get access to the receiver. This password concept would also keep the system secure and prevent a neighbor – or hacker – from controlling a device in your home. "

Well, my question concerns even the 'specific repetition gesture sequence'.

For instance, if I'm on my couch and perform the gesture while my cat jumps on the couch arm between myself and the WiSee, what happens?

Or my cat is in the other room, but walks in front of the WiSee while I'm performing the gesture, what happens?

"To achieve a strobing rave effect with your home lighting, simply do cardio."

So this was basically an S-band radar with a home wifi router? Can we make a mosquito zapper lazer with it?

You could probably, it wouldn't work very well. Either way, Nathan Myhrvold, evildoer and all-around asshole, would sue you for patent infringement. I recommend making a Myhrvold zapper laser first.

We already did that in 2007 :) https://www.youtube.com/watch?v=wSIWpFPkYrk

Love this idea. However 94% accuracy is not good enough. You need to be able to trust your action will lead to the desired result. 94% is like Siri getting your whole text message right, but getting the recipient wrong.

A lot depends on what happens the 6% of the time it fails. If failure just means that it fails to recognize that there was a gesture, then 94% is probably fine. It's not that big of a deal to have to gesture twice 6% of the time to turn off a light or skip to the next track.

Failures where the gesture is recognized as another gesture would be a lot more annoying.

Just wondering, do standard wi-fi chipsets provide the raw signals needed to implement things depending on spectrum-analysis like this? Some super-raw mode? How about Bluetooth chips? Sound chips (for supersonic)?

it should have been "ultrasound" instead of "supersonic" ;)

Great; so next time I sneeze, I'll accidentally turn on the garbage disposal?

Seriously, though, it reminds me of the "house of the future" from one of those ubiquitous films they showed us in elementary school way back in the day.

Honestly, the most exciting aspect of this would be in being able to just have a passive scanning system that can identify where in the house my kids are.

Could be useful in prisons to keep tabs on the general population without having to use cameras.

This may not be something we want.

It's going to happen anyway.

probably not, home automation has been "right around the corner" for 60 years.

Home automation exists and is cheap. There's a huge niche industry built around it. The problem is that people don't actually want it as much as they say they do. That doesn't mean it doesn't exist.

I think they are still a bit too expensive for people. You can either get a whole system for several dozen kUSD, or single elements (e.g. smart outlets) for ~$100 each. The former is a significant expense people might not want to pay at once when buying homes, and the latter is a bit too expensive to buy one by one, because you require to have at least few of those devices before the system starts to get useful.

It's not nearly that expensive, though. A light socket is in the neighborhood of $15. A wall socket is less than $25. A transceiver is $30. For just over $100 you can get the controlling software, two wall switches, two light sockets, and a transceiver.

http://www.thehomeautomationstore.com/x10-home-automation.ht...

Oh, but don't use CFL bulbs with those wall switches or light sockets.

...and good luck determining whether the device is on or off.

That's not the point, and there are units which address those concerns. The point is, home automation is not "right around the corner", it's here today and it doesn't take a millionaire to have a smart home. The sticking point is not technology nor is it money, it's that people have realized they don't really want that.

The technology you refer to has not changed since 1975.

It is a stretch to call this smarthome technology considering the ban on incandescent bulbs, which completely eliminate X10s ability to dim or fade bulbs (because you are using CFL, and if you use a x10 dimmer on a CFL, you can burn down a house), and writing any sort of logic based on whether the device is on or off is not possible.

Now Insteon supports bi-directional communication and you can determine state, but you are looking at $45 per outlet or switch versus the $8 X10 stuff.

So, yeah, still out of most people's price range, unless you want to settle for over 30 year old technology.

It is already here if you care to spend some money on it.

I should have said "Home automation for the masses."

I know, I worked at a startup in the space. The people who are capable of spending the money will spend the extra necessary to get someone to install it for them. The people who would install it themselves can't afford even the basic systems, as you need quite a few devices before the things get useful.

The most profitable use we ever found for it was to provide light switches and thermostats in hotels, in places where the wiring in the walls wouldn't allow. That's about it.

For under $1,000 you can easily build a significant home automation system.

http://www.amazon.com/b?ie=UTF8&node=2315509011

One definition of cheap would be something like $0.10 for self contained electronics in a light bulb. Then the installation is practically an accident.

(I think home automation sounds like a neat idea, but I always end up trying to figure out $10 of value that it would provide...)

> $0.10 for self contained electronics in a light bulb

I guess we'll get there at some point, but I'd rather prefer a $.10 for a wall-socket adapter with electronics. Much more versatile. I'd buy truckload of these and connect all stuff through it. As for lights, why not $.10 light bulb slot adapter?

> I think home automation sounds like a neat idea, but I always end up trying to figure out $10 of value that it would provide...

Fun and hackability, obviously :).

That's sort of tangential to my point. There are reasons we might not want phased array radar systems picking up every move we make in the privacy of our own homes, independent of home automation applications, which is I presume what the post I was responding to was getting at. My point is that those desires of ours are irrelevant when it comes to whether or not the applications we don't want come to pass, because at some point there's going to be a financial incentive for someone to do it anyway.

It's your own phased array radar system--oooh such a scary word! It's an input device. If you're worried about it telling Google that you spent 30 minutes on the toilet this morning, then either monitor the traffic for odd connections or just block it from making any connections outside your LAN.

It's not necessarily your own. 2.4GHz goes through walls.

I'm nowhere near to be techno-apocalyptic, but I just thought that from a "1984" point of view it could be a tool to monitor activities without no permission at all.

And wearing a tin foil hat just makes you a better reflector!

Anyway, if you look up through-wall radar detection, most of the implementations use S-band radar, and it is probably safe to assume that it's already been figured out how to use wifi stations and devices already in place emitting signals to each other, rather than always needing to bring additional equipment.

With regard to privacy though, it seems like it would be data and computationally intensive to track the movements of people in a large number of houses.

Let's say that a hypothetical application could only detect velocities away from or toward the base station, but no directional information. With each movement you make it would build a set of hypotheses about your location. Then when you make another movement, it would need to branch out, and create an entire set of hypotheses for each hypothesis in the previous set, and so on with each movement. While the program could abandon entire branches when they showed you walking through a wall, it would still grow very quickly.

Though it is possible to mine data of cell phone locations and CC television cameras after the fact, it seems like (if my understanding of how the technology would work is anything near reality) it is simply too difficult to collect a lot of information about people's movements in their homes without a specific prior reason to justify the expense.

Anyway, perhaps rather than tinfoil hats, people worried about surveillance should hang lots of mobile sculptures in their houses to create confounding noise instead.

This strongly reminds me of A Deepness in the Sky, albeit the implementation is different and the resolution is probably nowhere near the fictional stuff.

If used right, this would disrupt home security industry so hard. It would be possible to get rid of any motion detectors, if I am not mistaken.

If I understand this correctly, you would really be replacing one sort of motion detector with another.

The article is talking about phased array stuff, stuff you can't do with stock wifi equipment. All the regular wifi equipment is just providing the "light" the special receiver needs to see.

So really, it would make it impossible to get rid of motion detectors. You could never really be sure that one isn't nearby.

In the UK, signalling alarm systems (those that ring the police and so are actually useful) require that 2 types of detection technology be used in the one senor. Microwave and passive IR.

Using these together significantly reduces the chance of a false positive than just using one type on its own.

So, you could use this to replace the microwave sensor in the dual tech sensors but you would still need the PIR.

This is so cool!! I wonder how far off it is to commercial deployment and what licensing plans they may have.

How would you get started with this technology or stay updated with this project? I'm keen on its prospects.

Bye bye Kinect. Guess we won't be needing you now.

They are tracking all our movements via Wi-Fi

Damn, I posted this on hacker news before you.

It is now on Techmeme, too: http://www.techmeme.com/130605/p19#a130605p19

What is is, why do I care?

Or are you just spamming?

Reply on news.ycombinator.com