mmWave radar, you won't see it coming
joshhorne.com
joshhorne.com
With machine learning improvements, identification and more are possible. From a 2021 paper on IEEE 802.11bf Wi-Fi Sensing, https://arxiv.org/pdf/2103.14918.pdf
> Indeed, it has been shown that SENS-based classifiers can infer privacy-critical information such as keyboard typing, gesture recognition and activity tracking ... since Wi- Fi signals can penetrate hard objects and can be used without the presence of light, end-users may not even realize they are being tracked .. individuals should be provided the opportunity to opt out of SENS services – to avoid being monitored and tracked by the Wi-Fi devices around them. This would require the widespread introduction of reliable SENS algorithm for human or animal identification.
Prior discussion of WiFi Sensing:
Jan 2022, https://news.ycombinator.com/item?id=29901587
May 2021, https://news.ycombinator.com/item?id=27123493
400+ research papers on wireless activity sensing, showing steady improvement in detection techniques, https://dhalperi.github.io/linux-80211n-csitool/#external
Today, specialist devices or targeted attacks can monitor human activity through walls and closed doors. But that's a world away from the ubiquitous transparency scheduled for commodity WiFi 7 in 2024. If regulators or lawyers don't step in, homes and some businesses may need RF shielding.
Also, a person is not a randomly shaped object. If the processing is specifically tuned to detecting and identifying people an awful lot of degrees of freedom can be eliminated, giving more detail in those features that do identify a person.
> We earned this year’s award for a product targeted to launch in the fourth quarter: our Smart Health Monitoring Light. Featuring a Wi-Fi, Bluetooth Mesh dual chip, the bulb will provide a number of features, including biometric measurement tracking of heart rate, body temperature, and other vital signs, as well as sleep tracking. By connecting multiple bulbs via Bluetooth Mesh and creating a virtual map across your home, this product can even help detect human behavior and determine if someone has fallen and then send for help.
whatever happened to that meme about common ccd cameras picking up uv images that make out details through eg swimsuits, but hopefully filtered and removed in software before making it to the images?
In any case why would we be required to don tinfoil suits every time we go to the airport?
It's like if your car seatbelts have a bug you issue a product recall. Same thing.
I already do that, because I image nebulae and galaxies and sometimes landscapes in infrared.
It is not the material that’s buggy, it’s new technology which can look under clothes and damage privacy that’s the problem.
And then there's the fact that this is a hugely international business, so unless we convince the whole world to do this, we'll be impacting supply lines and flexibility there too.
The whole undertaking seems like it would have an absurd scope - just to avoid adding IR filters that cameras need anyhow to achieve accurate color reproduction, and therefore most cameras already include!
It's much simpler to lock doors, and that prevents thieves 99.9% of the time.
Breastfeeding mothers would be collateral winners of that change for example.
As I understand more or less a solved problem on modern cameras such as on your smartphone... instead of a bulb you tend to have white LED for your flash that doesn't really give off useful (any?) IR, and there's better/more consistent IR filtering on the sensor.
https://www.theverge.com/2020/5/15/21259723/oneplus-8-pro-x-...
Many smartphones (used to?) exclude this filter on the selfie-camera because no matter how hot you think you might be, it's not going to be an issue.
If you want to check an IR remote for function, try with the selfie camera of your phone and not the normal one and make sure you are trying in a dark environment because the LED on the remote is going to be rather dim either way.
[0] - https://skyandtelescope.org/astronomy-blogs/imaging-foundati... [1] - https://www.canon.co.uk/cameras/eos-ra/
If it’s IR… I have a PVS-14, I’ll have to dig one of my wife’s swimsuits out of the closet and see if it’s transparent. I’m curious.
Does that mean many of this applications are outright illegal in the EU, due to not having a good way to handle the case where the user doesn't opt-in?
Most spying on users through websites etc. doesn't fall under it do it being formulated in the pre-internet age.
But would WIFI-7 enabled routers fall under it? It's not complete unrealistic that it might happen.
The think is this law is rather strict in how illegal spy devices are to be handled => destructive disposal. (Required, by anyone owning such a device).
That could be quite a big fall out.
This law did hit some IoT devices, like some toys with integrated voice recording.
If you've got the signal density to illuminate the stuff you want to look at there's no reason you couldn't.
Its perfectly possible to create feature descriptors for 60gig radar, and use it to image rooms, things and possibly people.
Its a privacy problem we currently are trying to overcome in AR. The problem is that because feature descriptors are so small, researchers assume that they don't contain PII. So its going to take a scandal before its taken seriously.
Given that most of FAANG are making some sort of 3d map, I'm sure we are going to get pictures of people's bedrooms leaking soon enough.
Facebook for example. It took a few years, but people are now wary of information being grabbed by them.
It is either very dishonest or extremely stupid for an expert to say mm waves can't be used to identify a person.
- https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9136646 - https://arxiv.org/pdf/2011.00636.pdf
That said, it doesn't mean that every mmWave radar could work as SAR.
You do also need information about where the sensor was located at each pulse, but it doesn't have to be extremely precise - there are algorithms that do iterative processing to accomplish motion compensation.
> what it sees is not personally identifiable
Then later on in the article:
> [the new HomePod could] [detect] the user's identity based on their gait, height, and physical profile
Uh-huh.
…did they read their own article?
https://www.researchgate.net/figure/Millimeter-wave-40-60GHz...
There's a huge difference between a phased array scanner and the transceivers described in the OP, but mm waves are absolutely not "privacy-preserving"
Creating a new job market: RF shielding installation.
TI's documents are a good starting point: https://training.ti.com/sites/default/files/docs/mmwaveSensi...
But it doesn't really go into the different schemes for sending out orthogonal waveforms from the transmitters, the design of 2d arrays, much of the basic signal processing (windowing, matched filtering), etc.
I was pleased to see https://hforsten.com/mimo-radar-antenna-arrays.html pop up on hacker news quite a while ago, which was very interesting. Goes into how to plot the beam width for a mimo array, with some nice python code. I want more, but I'm afraid I won't get it unless I read through a phd thesis. Any ideas/tips?
Is 5mm resolution not enough to identify a face?
What’s the difference between a 3d model of a person constructed from high res radar and a black and white photo?
The radar sounds even more identifiable.
Edit: Couldn’t you construct a mosaic and have much higher resolution?
And also remove their clothes?
With greater number of antennas something like what you describe is theoretically possible, but becomes cost prohibitive.
By limiting the number of transmitters / receivers, we can have an almost perfect privacy monitoring. For example, it would be possible to detect a attack / a fight in a public bathroom, while not exposing anyone.
1. https://www.mouser.ec/datasheet/2/405/1/swru521c-1954464.pdf
For example, I have a CDM324 24Ghz radar module here on my desk. I set it up to 'watch' me type this comment from across the keyboard. This is an extremely simple module that I have powered by a bench power supply and the IF routed directly to the audio input on my desktop. It was sampled with audacity and amplified a bit to help with visibility. Towards the end of the recording I'm expecting a flat spot followed by regular motion followed by 'noise' as I pause motionless for ~10 seconds or so, then take about 5-6 exaggerated breaths, then resume typing.
This is with zero design, the wrong frequency for the job, and next to zero signal conditioning.
(post: i've included a zoomed in image of the 'motion demonstration' to show still/breathing/typing, then zoomed in on typing to show the detail, then a spectrogram and waveform of me reaching up to scratch my ear.)
Bonus: yo check out my soundcloud - This is what the doppler signal actually sounds like:
Scratching my ear - https://soundcloud.com/buckrunner2/scratch
Talking directly at the module - https://soundcloud.com/buckrunner2/talking
If we limit the number of transmitters / receivers to 3/4, the image reconstruction becomes impossible, similar how you would not really make a real photo from a few color sensors stationary looking at a scene, while a linear array of them that moves around an object would make a perfect scanner. It's a non-ideal analogy, but the best I can offer.
1. http://mt-fedfiles.blogspot.com/p/tsa-frequency-updates.html
2. https://www.3dbodyscanning.org/cap/papers/2017/17263mcmakin....
Gait analysis is excellent at identifying people, and that's projecting from a 2D image at a distance.
Anything which can draw a moving voxel cloud around a human is going to figure out who that human is eventually.
Plenty of real world privacy exists on a spectrum. My family knowing I'm in the bathroom is a far cry form them pointing a camera at me.
If stores track you "anonymously" with this ... Who cares? It's less invasive than the ubiquitous security cameras they have pointed at you.
If you have ideas how to reconstruct an image with 4 receivers / 2-3 transmitters, please, buy a dev kit from TI (they cost <$200, https://www.mouser.com/c/?q=mmwave) and show us a demo.
Just like with browser pinning, we have a variety of signals, which combined together give a very high rate of correct identification.
mmWave can be used to identify unique traits http://www.cs.ox.ac.uk/files/10889/%5BDCOSS19%5DmID.pdf
walk without rhythm ...
Could be a good opportunity for home Faraday cages.
The difference is now the neighbors WiFi can scan my house and see me inside it without me using WiFi.
It has become almost impossible to participate in society, without every aspect of your life being recorded in some way.
My Bank sells my transaction data to third parties to track my credit history and sell advertizing - they do this without my consent, or rather if I disagree, they will not have me as a customer. I could switch banks but it's basically the same everywhere. All I wanted was a place to store and transfer money.
My phone company sells my location and call data the government and to marketing companies - they do this without my consent, or rather if I disagree, they will not have me as a customer. I could switch providers but it's the same everywhere. What I actually wanted was to be able to make phone calls and have data connectivity on the go.
My ISP sells which websites I visit to marketing companies - they do this without my consent, or rather if I disagree, they will not have me as a customer. I could switch ISPs but it's basically the same everywhere. All I wanted was to have data connectivity at my home.
My airline sells my flight data to the government. They do this without my consent. There is no alternative. I just wanted to visit some relatives.
The government tracks all my phone calls. They do this without my consent. There is no alternative. I just wanted to call my friends.
The government tracks where and when and whom I send emails. They do this without my consent. There is no alternative. I just needed to communicate for my business.
When I go out in public, my image is records by a multitude of cameras, by a multitude of actors. They do this without my consent. There is no alternative.
My licenceplates are scanned, depending on my locality, more or less regularily. What I wanted was to make going around a bit easier over longer distances. There is no alternative.
My medical records are ... well there's something called HIPAA, but the reason it has to exist in the first place is that some seedy actors are trying to get their grubby little hands on this most private information of mine. I do not consent to this. I am sick. There is no alternative to medical care.
I go out in public and depending on where I go, cameras are being equipped with all sorts of biometric detection capabilities. All I wanted to go out for a drink, suddenly I am in some database.
I go online to browse the internet. No further explaination needed.
All I wanted was to have wifi connectivity at home. Some asshole has crammed functionality into the standard for wireless connectivity, that makes it possble to track where my body is. There is no alternative.
And so on and so on and so on.
All of these things taken together create an information asymetry. One by one they aren't neccessarily that terrible, but by accumulating all this information - for which there is constant pressure by the more authoritarian elements of our societies - creates a digital me, that absolutely has an effect on my day to day life.
Easiest example is my credit score, I am sure one can think up reasons to justify it's existance. But the point is, that this device exists as an incarnation just for me - without my consent at all. It's not that I need to be very important to have a credit score. I just have one by default.
It does not make me feel "important" to be tracked. It makes me feel oppressed. It's the opposite of important. Being treated like this, decisions being made about me by some usually invisible faces, makes me feel like cattle. I am being treated as a good that can be sold, and from which value can be extracted, in other ways than the exchange for goods and services for money. I am the commodity itself. By default I am presumed guilty, and evidence is being collected against me, even if I will never violate any law of society.
My right to live as a free interdependent member of society are being violated by the people who think they can use me and the data they can glean about me for whatever their purpose is - and they do this without even considering my consent.
Information asymetry enables control. It's the anti-thesis to freedom.
It has nothing to do with feeling "important" and the choice you are implying people have is not a free choice. It's forced.
Well said.
Disclaimer: Work at Google, but not on Pixel 4.
Intelligent fall detection using TI mmWave radar sensors https://www.youtube.com/watch?v=lsIo_HIk4GY&t=93s
3D Occupancy Sensing For Home and Office https://www.youtube.com/watch?v=9EDhJQbLNyo&t=83s
As you can see the radar doesn't reveal too much, while still being able to discern a lot of what is going on in each scene.
No, you can't see that, because it's a video of a specific visualisation of the data from one specific sensor.
If it can see through walls and measure biometrics like heart rate, mass and body type, build a 3d point cloud of limb positions and measure vibration, then it's more privacy invasjve than a camera, not less.
For analogy, take https://en.wikipedia.org/wiki/RANDU: it was very popular in its day, and its distribution looked good to the naked eye, but plot it in more than two dimensions and you see that it’s actually extremely terrible, failing spectral tests badly.
The information has been captured; the fact that one visualisation of it doesn’t expose that information is irrelevant and gravely misleading (as in: I don’t want someone that hasn’t already realised this working on this kind of stuff, because they’re dangerously ignorant and/or naive; I would rather they stop, and go and learn about the risks in detail before continuing).
if the data were leaked, would you rather have some of your vitals and point cloud data exposed, or a video of you taking a shower?
Airport body scanners use millimeter wave radar, and they use software to mask the image so it isn't personally identifiable. There are even passive versions that just use background radio waves in a compact form factor:
https://www.qinetiq.com/en/what-we-do/services-and-products/...
> This work explores the possibility of countering CSI based localization with an active device that, instead of jamming signals to avoid that a malicious receiver exploits CSI information to locate a person, superimpose on frames a copy of the same frame signal whose goal is not destroying reception as in jamming, but only obfuscate the location-relevant information carried by the CSI. A prototype implementation and early results look promising;
Bullshit!
[0] https://github.com/PeerHerholz/BIDSonym for example
Is it sensitive to noise or interference?
The VR headset application appeals to me. If headsets could be dumb terminals they would be a lot lighter and untethered wouldn't suffer from performance loss.
But I imagine for 60GHz WiFi it's probably pretty power hungry.
At this point there's not much noise in the mmWave spectrum (it won't be competing with 2.4GHz and 5GHz anyways). And mmWave doesn't travel that far which is one of the main reasons the FCC unlicensed it in the first place.
Might be worth keeping an eye on UWB for AR/VR headsets https://www.embedded.com/wireless-transceivers-use-uwb-for-l...
This is going to be in every home in 10 years embedded in phones, TVs, smart speakers. All public venues. Trams, buses, subways. Everywhere. But most importantly homes. Every. Single. Person. 24/7. Breathing rate. Abnormal motions. Current location. Current activity.
Maybe, just maybe if we raise a some awareness we can curb the proliferation a bit.
Surely I can't be the only one who sees this as a nightmare.
The construction industry would hand over wheelbarrows full of money for a BLE phone-paired puck that located studs and pipes behind drywall. Someone could even sell a "pro" version that combined the data from multiple pucks to render a 3D point cloud of the inside of the wall on the user's phone.
https://walabot.com/see-through-walls
This seems like a pretty good review:
To be clear you probably wouldn't get a 3D view of the whole room, just a narrow window like a conventional stud finder.
If you want to splurge (or are willing to couple trips to the coffee shop), you could buy one (or two) of these:
https://www.homedepot.com/p/C-H-Hanson-Magnetic-Stud-Finder-...
I was tipped off to these by the someone in the construction industry. He buys a few whenever they're in stock, so that he can have them in every toolbox, lose them, etc.
Note that, once you find a nail head, you can just leave the magnet hanging in place. If you have two, you can hang both, then use a straightedge or chalk-line to mark the point between the two where you want to make a hole. The fancy radar puck thing can't do that. (Maybe, someday, fancy AR goggles will solve that problem, but we're a long way off from that.)
There is a form of PIR sensing where it behaves like a thermal camera at a very low resolution (sometimes as low as 2x2 pixels)
that is not how "personally identifiable" works
as long as you can (re-)identify a person with it it's personally identifiable data. Even if identification is done by using a mmWave radar and some simple ML.
https://en.wikipedia.org/wiki/Radar_MASINT#Non-Cooperative_T...
Humans come in greater variety than airplanes but recognition, especially with things like gait analysis, should be quite possible. Now, I'm skeptical that a photo of a person contains enough information to allow for mmWave recognition though, but I'd assume video does.
This is the optimistic case. Perhaps more likely is that 'ambient' computing will smother us in surveillance, ads, 'nudges', restrictions (you seem tired or drunk, you may not drive), etc.
Call me a cynic but since the power button stopped being a power button I have trust issues. In the end we'll have mmWave chips our company devices that can tell middle managers if we've been at our desk for all 28,800 seconds of the work day and exactly how hard we've been working, that's a more likely outcome than saving a kid in a hot car.
Anyone with the desire could set up such systems today. But they don't. Because they're expensive, inaccurate, and ultimately not all that useful in the social context. There are places that do employ such monitoring, but they're a corner case.
Trust is a core commodity in the workplace. It's usually more profitable to establish a trustworthy workgroup to accomplish some goal than it is to accomplish some goal WHILE monitoring the workgroup's every move for infractions.
Don't amazon warehouses already monitor workers in this much detail? Just yesterday I read someone talking about how managers get reports containing, amongst other things, how often a person was standing still. Lots of companies squeeze their employees like this, especially low paid employees.
The basic idea is your phone becomes your life coach.
This "privacy protecting" radar (which does not seem privacy protecting at all, just avoids leaking images which is just bad PR avoidance) looks like the same direction.
I agree this will happen, I suspect Apple is not only in the best position to exploit mmWave hardware but already well down the road in related interaction design.
Apple Watch already offers limited physical sense-based life coaching in the form of hand washing duration tracking and feedback.
Apple Watch’s assisted interaction with UI that quietly debuted in iOS 15 was an important set of physical sense-based interaction primitives in production.
They suggest to me the company has likely progressed capabilities sans mmWave.
And yes, I am sure many companies are thinking (or at least have people in them thinking like this). A few issues are the legal and cultural framework (I am quite happy for my personal data to be shared with accredited Health (ie NHS) researchers, but not with the Apple Watch product manager)
Epidemiology at this level is going to have transformative effects, but it needs to be trusted. I mean the only difference between a utopia of guided humans and a surveillance state is who gets the data and what they do with it. the people in the free country abs the ones in the oppressive totalitarian state will be wearing the same devices with the same radar.
I do think Apple captures plenty of meta data, but I do not know how much physical motion is getting sent back. One of the goals of the secure enclave was to store biometrics. And despite some notable concerns, the company has generally aligned its business model and branding to be privacy focused.
I can't think of another large company I'd trust more to handle this kind of information.
>Epidemiology at this level is going to have transformative effects...
The transformation is already underway. When Apple delivered Google Maps on iPhone with its pulsing blue dot, those with the technology instantly leapt ahead of those without it. I wonder how much stress has been relieved just from people knowing where they are.
I agree the technology can be misused, even to oppress. But this is true with any tech.
My concern is more about the disparity of those who have this tech and those that don't. The enhancements could start to show in a matter of years and quality of recommendations could coalesce into what will look like a new plane of daily living.
We produce high-end FMCW radars (http://dopplerradars.com) and I have been looking for ways to align us more with Tech and less with Defense.
https://ans.unibs.it/projects/csi-murder/
> Imagine that someone wants to illegally track the position of a person inside a laboratory, for instance to measure how much time is spent doing different activities at different desks, as depicted in the upper picture. How much effective can this attack be? ... With CSI-MURDER, the localization becomes impossible because results will seem random, thus preserving the person privacy without destroying Wi-Fi communications
Make it all locally manageable, and make it work with HomeAssistant, so I don't have to deal with it "calling home" or shipping data outside my LAN.
I'm not a huge fan of the current crop of presence detection, and mmWave seems like just the thing. Eventually, person detection would be fantastic, but it MUST NOT call home outside the LAN. I don't want that junk going on the internet in any way, shape, or fashion.
I need a presence detector that's able to detect children near my pool.
I need a presence detector that can, via gait detection (or other), detect when a specific person crosses a threshold (e.g. autism or Alzheimer's with a tendency to run off)
I'm sure the OEMs would be interested, but I'm looking for something I can retrofit.
The radar isn't using the entire 0 Hz to 60 GHz band, it will be using something like a few GHz or sometimes much less. i.e. 60-64 GHz
Also much of the processing isn't done on general purpose processors but in FPGAs or ASICs which can do lots of the parallel computation very fast.
For comparison 802.11ac has a max of 120 MHz channels, and that's cheap and everywhere these days (but runs on 5 GHz).
There are lots of basically tricks to reduce the amount of data in successive stages so that the processing is quite possible.
E.g. for 802.11a/n/c the CARRIER is in the 5.x GHz band but the modulated signal BANDWIDTH will be one of 20, 40, 80, 160 MHz around a central frequency [0]. The 'base' 20MHz bandwidth at best performance (signal to noise ratio) will be able to modulate (encode) at a raw 54Mbits/second.
Intel publish [1] an easy-to-read set of tables that show the relationships between standards, frequencies, bandwidths, MIMO, and modulation schemes.
[0] simplified - actually there is a (frequency-hopping or direct sequence) spread-spectrum method
[1] https://www.intel.com/content/www/us/en/support/articles/000...
Wasn't the blocker regulations in India?
would be interested if you have any links about it
can't be evil > don't be evil.
Both views seem to miss the mark. The crypto folks see mathematical certitude while ignoring the dynamics of the social/legal context in which it must operate. By contrast, those who argue that "artifacts don't have politics" would absolve technologists of an ethical duty to consider how their products impact society.
In short, privacy required both technical and legal protections as well as a society that demands it as a fundamental human right
[1]: https://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis
But I don't think there is much risk in gesture control.
Why using the name of a cryptocurrency for something that has nothing to do with it??
Ha, haha, hahaha!
It can measure your breathing, heart signature, gait, etc. With machine learning this data will certainly be enough to identify you.