Espargos: ESP32-based WiFi sensing array | Hacker News Reader

Espargos: ESP32-based WiFi sensing array

espargos.net

199 pointsleoedin35 comments

This project uses an array of ESP32s and some clever calibration techniques to do pretty accurate localisation of WiFi signals.

There's an excellent companion video here: https://www.youtube.com/watch?v=sXwDrcd1t-E

This educational visualization will be helpful as WiFi 7 Sensing comes to consumer routers and NPU/AI laptops, rendering home and business walls transparent to human activity like typing, breathing & more.

"Inside a $1 radar motion sensor", 100 comments, https://news.ycombinator.com/item?id=40834349

https://www.cnx-software.com/2022/08/08/esp-wifi-csi-detects...

> Espressif claims it can also capture subtle movements caused by small movements such as breathing and chewing of people or animals in a static environment.. works with all ESP32 series microcontrollers including ESP32, ESP32-S2, and ESP32-C3, and does not require any changes to the hardware

24Ghz/60Ghz mmWave sensors for finer resolution, https://www.dfrobot.com/product-2795.html

I've got a 60Ghz mmWave sensor mounted above my bed (at the suggested distance and orientation) and it has been terrible at detecting sleep / heart rate / breathing. Reasonably good at detecting presence though. Maybe there's some tuning/calibration can be done but I've not seen anything about that.

RF sensor has to be coupled with activity-specific ML models for detection. Google Scholar has hundreds of academic papers on wireless sensing with machine learning, targeting specific activities. Commercially, Google Nest Hub 2nd gen includes mmWave sensor for sleep tracking.

Hmm, seems like I’ll be able to replace my cantenna with this to find malicious IoT devices spamming the airwaves with DeAuth attacks…! Imagine being able to catch crooks using Flipper Zeros to defeat wireless cameras…can track them in real time ;)

> Imagine being able to catch crooks using Flipper Zeros to defeat wireless cameras…can track them in real time ;)

One of the reasons I run Ethernet to everything.

Also, I assume you work in security if you deal deal with malicious WiFi devices and sophisticated crooks?

Wired is definitely the best! But sometimes wireless is the most feasible and quickest way.

+1 for cantenna reference.

That is very nice! It reminds me of Usman Haque's WiFi Camera project from 2006. That wasn't anywhere near realtime but it did use wasabi-pea cans

https://haque.co.uk/work/wifi-camera/

I miss tech art that doesn't take itself too seriously.

I am very interested in knowing how they are able to get such precise measurements with an ESP32.

In their chart there is a time of arrival difference that has nano second accuracy. The ESP32 runs at 240MHZ max, and usually it takes several cycles to read certain values, so I really wonder how they were able to pull it off.

Please enlighten me.

Just a gas but maybe it's because they're using multiple esp32's in an array? But also I seem to recall something similar in the Snowden papers? But I get your point, given that the esp32 and all its design files are available from SPRESIF maybe they built a board that's running out of much faster clock rate? PS: sorry about the typos and miss takes but sadly I have to use Voice typing due to a recent traumatic injury.

> The ESP32 runs at 240MHZ max

Luckily for 802.11mc, host CPU speed is not used in the distance estimation.

As shared elsewhere in this thread: https://www.youtube.com/watch?v=sXwDrcd1t-E

TL;DW: the ESP’s WiFi driver provides per incoming packet the carrier’s phase and amplitude. All ESPs are calibrated relatively to each other.

Every WiFi chip needs to be able to synchronize to the received signal down to a few I/Q samples (the remaining synchronization error is accounted for by the cyclic prefix of OFDM). For example, with 40MHz of bandwidth, there is an I/Q sample every 25 nanoseconds (the wave travels 7.5m during that time), so the level of synchronization will be on that order. The ESP32 reports the time of arrival, the code for extracting a nanosecond-precision timestamp is here: https://github.com/ESPARGOS/pyespargos/blob/main/espargos/po...

Additionally, you can use the phase information in the estimated channel coefficients (CSI) to determine the more precise time of arrival. Very briefly, a time delay in time domain will be visible as a frequency(=subcarrier)-dependent phase shift in frequency(=subcarrier)-domain. Now synchronization is only limited by impairments like thermal noise and phase noise. In practice in nice lab conditions, the time of arrival accuracy is on the level of <<1m.

On top of phase differences between subcarriers (--> timing information), we also measure phase differences between antennas, which provides angle of arrival and, with multiple arrays, phase of arrival information.

You can go even further by measuring phase differences between subsequent packets, which provides frequency offset and Doppler (--> velocity) information. I have some nice results on that, but that's a topic for the future ;)

Thank you for your reply, I have been reading the code and there are some things that I am not sure to understand.

- The Espargos code uses rxstart_time_cyc & rxstart_time_cyc_dec for the ns timestamp calculation

- Both values are not documented in the Espressif code and just marked as /*< reserved */

- The timestamp parameter is documented but it is in "microseconds"

Is there anywhere I can find more documentation on the "rxstart_time_cyc" and "rxstart_time_cyc_dec" fields? Are these cycle counters from the wifi chip?

See https://github.com/espressif/esp-idf/issues/9843

It makes sense now. Thanks.

(unrelated to ESP32)

You can often get some sample timing resolution from a sampled signal by interpolating the result of a complex correlation. You start by searching the correlation output for the loudest hypotenuse lag, then take he worked tangent of the quadrature information of that lag to estimate the sub-sample event time

Earlier video: https://www.youtube.com/watch?v=sXwDrcd1t-E

(https://news.ycombinator.com/item?id=43062982)

Very cool! Website bug report (hoping somebody sees that): on my phone (iPhone latest iOS) the videos would automatically play in full screen when scrolling the page. Really annoying :)

Same for me. Though I just assumed it was some sort of annoying “feature”.

Oops, never tested the website on Safari / iOS. I will look into this at some point, opened a GitHub issue for myself so that I will remember: https://github.com/ESPARGOS/espargos.github.io/issues/1

Is this locked to WiFi spectrum, or could the antennas be swappen for whatever frequency band you want to target?

Say starlink frequency for tracking or The NOAA satellites?

ESP32 will be just 2.4ghz, and the bandwidth of the antennas will also limit.

Totally doable to make a wideband version with different hardware. This is called a phased array, which is common in radar applications (e.g. Aegis cruisers).

Espressif has a new dual-band chip that does 2.4ghz and 5ghz wifi. That isn't the subject of this article, but ESP32 is gaining some new wifi capabilities.

I'm currently experimenting with ESP32-C5 engineering samples provided by Espressif, so hopefully some future ESPARGOS revision will be 5GHz-capable :)

But yeah, as far as I can tell, the chip can only extract CSI from WiFi (unless there is some hidden raw I/Q sample debug mode or something).

This project is amazing! I know there's some prior art, but I feel like I've never seen anything wifi-related moving so... smoothly? (With that high of an update/sample rate, I guess?) as your visualizations do.

I do have one question that wasn't quite clear to me, in case you happen to see this. There's a bunch of mentions of Channel State Information, but Google doesn't give a ton of info on it. So I'm just wondering, is CSI a "client-side" thing provided by the ESP32 for any Wi-Fi connection, or is it some special protocol feature that the AP or client have to enable?

Most wireless standards transmit some kind of pilot sequence next to the payload. A pilot is a sequence that is well-known to both TX and RX, and which allows the receiver to estimate the effect that the propagation has on the signal. The channel estimate is then used for equalization, i.e. undoing the channel effect.

For WiFi, some of the pilot symbols are called L-LTF (802.11g+n) and HT-LTF (802.11n). Every packet contains them as they are needed for successful communication. The ESP32 can provide channel estimates based on L-LTF and HT-LTF for any packet it receives. No need to enable anything, you can get CSI from every packet.

Awesome! Thanks for the clarification.

While the esp32 is only going to be able to do 2.4 GHz. KrakenSDR can be used in a similar way to do spatial mapping of other frequencies. There was a passive RADAR project for it as well that I think was taken down because of ITAR. https://www.krakenrf.com/

I’d note that Xandem while a total flop commercially, does own patents on tomographic motion detection.

It’s a shame honestly, it was a really cool product.

The hackster article has a bit more technical infos: https://www.hackster.io/news/exploring-the-invisible-world-w...

I wonder if this can be used to aid in per room presence detection.

24ghz radar sensors are pretty damn accurate too (way better than the infra red based ones). About 10 bucks off aliexpress

e.g. count people watching TV for pay-per-view billing, https://news.ycombinator.com/item?id=43076335

Reply on news.ycombinator.com