Person-in-WiFi: Fine-Grained Person Perception Using WiFi [pdf] | Hacker News Reader

Person-in-WiFi: Fine-Grained Person Perception Using WiFi [pdf]

ri.cmu.edu

150 pointsRCitronsBroker67 comments

Using neural networks to solve inverse-scattering problems (like wifi scattering off a human body, for example) seems to have a lot of potential. The lack of phase-information (i.e. not just signal intensity but instantaneous phase of the EM wave) captured by traditional receivers makes this class of problems so difficult to approach since you are blind to a significant portion of the available EM information. Mitigating this by constraining your solution-space to 'reasonable' outcomes is practically very difficult... for a human. Very cool to see such a practical demonstration of a neural network seeming to accomplish exactly this.

WiFi radios are notoriously insecure. Let's connect them to AI neural networks! OpenAI WiFi 7 Sensing routers for everyone.

https://news.ycombinator.com/item?id=38466715

CSI counter-measures

http://www.orca-project.eu/

(2021) https://ans.unibs.it/projects/csi-murder/

> Imagine that someone wants to illegally track the position of a person inside a laboratory, for instance to measure how much time is spent doing different activities at different desks, as depicted in the upper picture. How effective can this attack be? ... With CSI-MURDER, the localization becomes impossible because results will seem random, thus preserving the person privacy without destroying Wi-Fi communications.

Great comment, seriously

I was looking into this recently, because I recalled seeing the video of this on Twitter a few years back and wanted to see if I could reproduce it. But the WiFi signal strength tools they used appear to be abandonware, and according to this researcher they may not have even accomplished this task at all: https://medium.com/@tsardoz/researchers-misrepresenting-the-...

Archive link: https://archive.is/MnOv0

Actually I last month reviewed also a wifi sensing paper and somehow it also did not really elaborate on the validation scheme. It seems that still some better standards need to be estaished. Often it is to good to be true.

IEEE standardization work is nearly done. It works. Mass commercial deployment in 2024-2025.

You don't have to wait too long this functionality has been baked into WiFi7 with SENS 802.11bf

I hope folks understand what this means - there will be an IEEE standard that makes using Wi-Fi sensing, to some degree, hardware agnostic and thereby drives it into mainstream.

Wild. These seem to be decent intros.

https://arxiv.org/abs/2103.14918

https://arxiv.org/abs/2207.04859

Can't wait until my smartTV starts pausing ads when the wi-fi sensing determines I'm on the toilet and not watching.

Does anyone know of a way to replicate these results at home, even at lower fidelity? I've never found code I could run and it seems to always involve extra bespoke antennas/drivers.

This is similar but should be enough to get you started! https://github.com/espressif/esp-csi

WiFi Sensing code for ESP32 WROOM, https://wrlab.github.io/Wi-ESP/ and Intel 5300, https://dhalperi.github.io/linux-80211n-csitool/

great, now what are the applications and how will this be commercialized into a torment nexus?

The same way police go directly to Amazon to get home security camera footage from Nest devices, they will be able to work with your ISP (if you rent a modem-router-ap) or cloud-based-network-gear provider to pull this data from people's houses when serving no-knock warrants or investigate crimes.

You mean ring? Nest is a google property. Doesn’t even use aws, so pretty sure Amazon would just give the police a dumb look if they were asked for footage from nest cameras. Also last I checked nest was one of the better players in this regard requiring warrants and had rigid public data retention processes in place to limit what could be pulled.

https://xkcd.com/538/

but seriously, ISPs can already provide MAC addresses of all active client devices at the building at any time, no need to tell the dog from the owner puzzle by sci-fi tech.

There's got to be a limit to that. Most consumer home routers perform NAT, so the ISP should only ever see the MAC of the router's WAN interface.

The exception would be if you're running an ISP-supplied modem/router/AP combo unit, but in that case it should be no surprise they can see virtually everything on your network.

Another exception is bridging IPv6.

Another exception is logging devices that actively scan for networks.

There are probably more exceptions that neither of us have thought of. Bringing a device into your home that is owned by someone else introduces all kinds of security side-effects. When that someone else is a major player in the telecom/tech space, you should be extra-weary of it.

99% no tech people just plug in what they get from ISPs, especially fiber where you also get IPTV plan over WiFi/fiber too.

I think that this would primarily have military/police applications. Being able to tell in real time where the people in confined space are just by hacking their WiFi sounds invaluable.

No hacking required.

A passive receiver outside a wall can sense reflections passing through the wall, from the transmitter inside. Applies to neighbors and adversaries.

Imagine Streetview sensing cars, capturing not only home exteriors, but interiors.

Attaching a receiver to a wall outside might be much more challenging than hacking in certain situations.

No attachment necessary, only antennas. An attacker could target devices inside or outside the building.

WiFi connected surveillance cameras are pervasive. What happens if they can "sense" through the walls of buildings?

Hundreds of public research papers include terms like these:

  human-to-human interaction recognition
  device-free human activity recognition
  occupant activity recognition in smart offices
  emotion sensing via wireless channel data
  CSI learning for gait biometric sensing
  sleep monitoring from afar 
  human breath status via commodity wifi
  device-free crowd sensing

No mention of hive minds.

Missing (2019) in the title.

Yes, the paper linked to by firebirdn99 is much more recent and shares an author.

I remember seeing many posts about this in the last few yrs, one - https://news.ycombinator.com/item?id=34423395

Well the paper is from 2019.

I find these applications intriguing for VR/AR/XR, but there are also the privacy concerns. I can see internet providers and big tech really wanting to leverage this data for profit

the privacy concerns are really something. Background emission is pretty much as abundant as ambient light these days, at least within the radius of day-to-day living. even in less developed countries. Western signals intelligence gathering already reached critical mass, the decision to deploy dragnets collecting the tons of data we all produce every day, has already been made.

Just frightening to think about the never ending increase in detail and variation when it comes to the data to be collected.

but look, 300 bucks for the privilege of inviting a self-propelled LiDAR scanner, complete with internet connection and non-optional user registration, into your domicile! It can vacuum too? Sound like a deal to me!

Though only tangentially relevant, I can recommend Valetudo as a way to take control of your vacuum's firmware.

thanks mate! just forwarded this to a friend of mine. he will be delighted for sure

I'm less worried about the govt surveillance and more so about abusive advertising, though I wonder how much additional insights it will really provide. The amount the data brokers and scientists can infer already is pretty astounding

Why not both? Both are bad and both should be minimized.

The government has ample ways to get the same data via other means and probably more sophisticated tech. I would imagine they are well beyond this

I also don't do things that would have them watching or worrying about me, so not really concerned personally and only see the upside to the government / military having this capability, with appropriate oversight (which I'm sure many will imply is non-existant)

There is a valid product offering for ISPs, monitoring people for medical emergencies. I suspect this will be how it gets accepted by many people

Would you say this is any different from the classic "I have nothing to hide" argument?

Edit: https://en.m.wikipedia.org/wiki/Nothing_to_hide_argument

not sure, I don't want the gov't to have unfettered access or capabilities, they need oversight and to provide transparency. I'd rather see them put out the reports of how many times they use it, rather than relying on companies volunteering the same information as it applies to them. At least some of them do in the absence of a self-reporting gov't agency

generally, there's a balance and it tends to oscillate around that depending on larger events and trends

WiFi 7 sensing is not gov specific. It's going into commercial routers and laptops.

Speech surveillance has lead to self-censorship of speech.

Motion surveillance can lead to self-censorship of motion.

I think my speech has been self-censored more by the woke movement than the government. Social norms or fads seem to outweigh anything the gov't would get away with, at least in the US

WiFi and other through-wall radar sensing has been sold to governments for more than a decade, and research code/firmware/papers have been online for 5+ years.

WiFi 7 Sensing expands the list of ecosystem participants to hundreds of millions, i.e. it's about scale and ubiquity, not any particular actor.

No one can predict what it means to abolish opaque walls. Observations are going to alter the observed.

I wonder if at some point the amount of data each individual produce would be so large and contradicting that it would be almost unusable by all but the most advance tech department and with great resources.

No, it's the opposite. Dedicated silicon (TPU, NPU, Intel/AMD client AI) blocks on commodity processors are lowering the cost of edge processing of high-volume raw data into low-volume, high-value signal for centralized aggregation.

Precision of WiFi sensing resolution increases at higher frequency / shorter wavelength, e.g. upcoming mmWave standards.

Why should privacy be the objective of the scientist?

I mean, why should any scientist think about ethics? [INSERT $GOLDBLUM_QUOTE HERE]

Also, when you start to invent new technology, you're not just a dispassionate objective scientist pushing the boundaries of knowledge. You're an engineer whose products are also engineering society.

"Your scientists were so preoccupied with ..."

That's a Jurassic Park quote? Wow!

I'm not faulting the the scientists here, they aren't the first to do this. I remember seeing similar from a US university research a few years back. This reminded me about the latest wifi-6E(?) that's going to make this easier for ISPs to obtain similar tracking capabilities.

I for one always own my wifi/modem stack rather then renting the one from the ISP

Because observers become part of science experiments.

It's where the truth lives.

Wasn't there something like this a while back where somehow had designed an "xray scanner" that used peoples wifi routers to do it?

The one I recall trained on wifi and video at the same time, then did inference on only the wifi, with good reports. It was one of those research papers that all the media covered

1600+ papers cite the Intel paper (2011) with firmware for CSI sensing, https://scholar.google.com/scholar?cites=4354757827101869878

At Intel, Bob's contributions to Northwood were credited as "the first x86 architecture that can no longer be comprehended in its entirety by a single individual." Basically it was his Swan Song proving there is a limit to how far you can push x86 CISC before it implodes.

Wrong article

I know! It was too late to delete after I realized it.

I want something like this for accurate room presence detection in my house (for lights etc) that doesn't require dozens of PIR sensors or similar.

My dream is that the lights in my long hallway gradually dim up in front of me and back down behind me as I walk.

WiZ lightbulbs actually surprised me with this feature - they call it SpaceSense. You need two of their lightbulbs in a room to enable that feature: https://www.wizconnected.com/en-us/explore-wiz/spacesense

I'd settle for just "is there a person in this room?"

Taking even 30 minutes to converge that answer would still be great for lighting automation.

There are mmWave (5.8-60GHz) human presence sensors, https://smarthomescene.com/reviews/best-mmwave-human-presenc...

Aqara FP2 from China can isolate humans and room zones, https://smarthomescene.com/reviews/aqara-fp2-human-presence-...

Yeah, the mmWave sensors are a good step forward for detection presence compared to PIR, especially being able to define behavior based on multiple zones, although the latency is extremely high compared to PIR/ultrasonic. I was hoping to replace my PIR sensor with the FP2, but in the end I’m having to run both — PIR for instant on lights when entering room and mmWave for not leaving me in the dark when I’m at my desk working.

With that said, even if we mmWave sensors see significant improvements to latency, cost, and power consumption, I’m fairly convinced that it alone will never give me a completely satisfactory solution to presence. My garage/home office situation may be unique (or not), but I’ll always need a synthesis of multiple inputs/cues (SPL monitor, active BT connections, contact sensor activity, etc). Gotta love the lights shutting off abruptly while soldering audio equipment b/c I was laying down on the floor in the corner, and rather still, instead of doing normal human things. As with all good problems, just throw another abstraction layer at it :)

I never understood why the Aqara mmWave sensor doesn't also include a powered PIR sensor in it.

This work is very similar to Srinivasa Narasimhan's Group in CMU, surprised he is not one of the co-authors.

Not only is this coming in 2024 Wi-Fi routers (IEEE 802.11bf WiFi 7 Sensing), but the Intel Meteor Lake SoC includes a dedicated IP block for "AI Sensing" to process sensor data from cameras and radios. The former lead of the Intel client AI team has moved to AMD, and Ryzen already has an AI IP block, so AMD may follow.

While radar has been around for decades, research into WiFi doppler imaging was jumpstarted by Intel's release of custom wifi firmware around 2010. A decade and hundreds of research ("device free wireless sensing", "CSI imaging") papers later, there is now open-source code for doing this on sub-$20 devices. In the mid 2000s, similar imaging was used in WoT conflict zones to locate adversary humans by "seeing through walls".

Radio spectrum and devices using that spectrum are managed by national regulators in each country, who can set policy for authorized applications. Until national regulators step up, the only defense against passive through-wall, Wi-Fi Sensing is EMF shielding for the walls, roof and floor of specific areas in homes and businesses. Otherwise, "The Transparent Society" (David Brin) is incoming.

https://lkml.org/lkml/2023/2/12/314

> Intel Visual Sensing Controller (IVSC), codenamed "Clover Falls", is a companion chip designed to provide secure and low power vision capability to IA platforms. The primary use case of IVSC is to bring in context awareness. IVSC interfaces directly with the platform main camera sensor via a CSI-2 link and processes the image data with the embedded AI engine. The detected events are sent over I2C to ISH (Intel Sensor Hub) for additional data fusion from multiple sensors.

https://www.techpowerup.com/276114/new-intel-visual-sensing-...

> The company didn't detail how it goes about this, but technologies already exist to combine visual input from the PC's cameras; radio from the PC's antennas, audio from its mic array; to form a picture of its surroundings.

https://community.intel.com/t5/Blogs/Tech-Innovation/Client/...

> With an initial focus on respiration detection, we hope to extend the technology to detect other physical activities as well. Intel Labs will demonstrate an early prototype of breathing detection ... The solution detects the rhythmic change in CSI due to chest movement during breathing ... The respiration rates gathered by this technology could play an important role in stress detection and other wellness applications.

https://arxiv.org/abs/2103.14918

https://arxiv.org/abs/2207.04859

Sort of related: Walabot

Reply on news.ycombinator.com