Show HN: Find3 – An indoor positioning system for smartphones and laptops | Hacker News Reader

Show HN: Find3 – An indoor positioning system for smartphones and laptops

github.com

265 pointsqrv3w58 comments

Our (used to) lighting division is selling a product where we use our LED-lights to track phones indoor. Always thought it one of the coolest applications of our tech. Using the fact that we can control LED frequencies invisible to the human eye but visible to a phones camera to pinpoint devices and provide indoor navigation [0]. Accuracy was tested to be 30cm (when combined with a few other factors).

[0] http://www.lighting.philips.com/main/systems/lighting-system...

Roosendaal by any chance? I remember visiting the lighting r&d department with school when I was young. Was one of the things that inspired me to go into tech! They demolished the entire factory now ...

Not exactly sure where it was developed (I am working in the heathcare division). Today most of it is in Eindhoven. I would guess the R&D divisions in Eindhoven developed it but that is pure speculation.

ASML is in Eindhoven. I worked there for about a year, and they used to be a part of the Philips R&D group.

So, that makes total sense.

That is a very interesting approach to indoor positioning. The idea of just installing a light bulb seems very cost-effective.

When you say, “used to” did you mean that the product was discontinued and/or isn’t available for purchase anymore? Or perhaps Philips sold the technology?

I’d like to learn more, will use the Contact Us form on the page you linked but if there’s someone you could recommend that would be much appreciated.

I jokingly said “used to” as Philips split off the lighting division. It is now rebranded to Signify. Technology will still be sold under the Philips brand.

The indoor positioning is also still being sold as a product. As far as I know we are doing some mega stores and expanding.

If the contact form doesn’t work DM me and I can see if I can get a direct contact for you. (Added email to bio)

This is kinda unrelated, but could you tell a bit more about the R&D departement inside Philips?

The reason for asking is that there are a few companies in The Netherlands that I'd love to work for, and Philips is one of them. (Other companies would include the NS or Thales) Furthermore R&D does sounds like something that I would love. However being mainly a software engineer with a touch of mechanical engineering I'm not sure if I'd be a good fit.

I know that Philips is only hiring interns at the moment, but I assume that most departments won't differ much from other 'big' Dutch companies.

Sure. I’d be happy to help though let’s not go off-topic in this thread. Feel free to drop me an email, see my bio for the address.

Hi bastijn, just following up. I don't see your email in your HN profile but I would like to take you up on your offer to get in touch with the appropriate lighting folk at Philips. I haven't heard anything back from my attempts via their online form. My email is in my HN profile if you don't mind getting in touch. Thank you.

Hi. Sorry removed it recently again as I thought we had it all covered :). Your email is not in your bio either so I added mine again. Feel free to contact!

That sounds really cool! This requires the phone being out of the pocket though, right? This would be great for way-finding, but maybe harder for home-automation.

Yes I believe it does, at least to stay accurate. The idea is it is used as navigation and to show location aware services indoor which anyway requires you to have your phone out. Any moment you pull it out it will sync your position.

In addition we use the system to make smarter offices [0]. The lighting grid is used to track where employees are, and will be using ML. All to reduce the CO2 footprint by smartly heating only the right areas of a building at the right times, and do the same with airconditioning, lights, etc. see link below.

[0] http://www.lighting.philips.com/main/cases/cases/office/edge

I wonder how many animals are exposed to intense lights and sounds and radio signals that humans don't detect.

Perhaps this is the reason for butterflies and bees to decline.

I have often wondered the same. It must be torture for animals with a higher flicker fusion threshold. The Wikipedia page seems to suggest that your threshold is related to your metabolic rate.

https://en.wikipedia.org/wiki/Flicker_fusion_threshold

I have created a find3 beacon/client for the cheap ESP8266/ESP32 microcontrollers yesterday and released it to the public. https://github.com/DatanoiseTV/esp-find3-client

The ESP8266 uses WiFi and submits the BSSIDs and their RSSI to the server.

The ESP32 version uses BLE probes additionally, which potentially improves accuracy.

Any feedback is welcome. This is an early hack.

The byline says "High-precision" but I could not find any benchmarks or scientific analysis. What does high precision mean here? How stable is it?

What about privacy of the users? Does the server know where everyone is?

Yep. Needs to answer what is the precision?

The actual precision you get depends on context, as this system utilizes surrounding signals (Bluetooth/WiFi/etc.) in the given environment to classify the location. Generally rooms/walkways are easily distinguishable. Pieces of rooms are also usually distinguishable because of smart devices people generally acquire nowadays (e.g. a Google home at one side of the kitchen is sufficient).

I say high-precision because I compare it to GPS, but if you compare it to round-trip time (RTT) measurements which get cm-level precision from specialized hardware, it is an order-of-magnitude behind.

The public server knows the mac addresses of devices that they send to it, so in a sense it does know where they are (because those can be reversed with public wardriving databases). The public server does not have authentication, so if someone figures out your "family name" then they can see your data. I suggest using the public server to try FIND, but not to use FIND so I have instructions for setting up your own system. [1] I'll make this more clear in the docs.

[1] https://www.internalpositioning.com/doc/server_setup.md

Just be aware that the public server in the app has no access control. Your location will be published.

You probably really want to set up a private server to use this in practice or even try it out.

Otherwise looks promising. A nice addition to my home setup

Thanks. I will add something across the system to highlight this. I want to encourage folks to run their own server [1] and use public cloud just for playing.

[1]: https://www.internalpositioning.com/doc/server_setup.md

Edit: For those who finished playing and now want to delete their data from the public server, I just added a DELETE endpoint: https://github.com/schollz/find3/blob/master/doc/api.md#dele...

Hey, I remember this project! I had the chance to give a bit of advice to the creator, Zach Scholl while I was working at Duke's Co-Lab [1]. It was cool then, and looks like it's only improved in robustness since (and it sounds like HN has plenty of ideas for PRs).

Although I no longer work/learn at the Co-Lab I'd love to plug the value of these sorts of programs at schools. They give encouragement with money and publicity, and most importantly, tools (VMs/APIs) and advice ("why won't this compile?"/"where do we start?"). I had a chance to see and help with a lot of projects that wouldn't have otherwise been possible to get built. Hopefully it helped a few people get into hacking that wouldn't have otherwise too!

[1]: colab.duke.edu

Thanks! Yeah I think I would've never gotten the idea out of my idea without the support from Co-Lab.

Haven't read through the code yet, but the only thing I need to know is if these guys know what Rayleigh scattering is. If not, this probably isn't going to work so great.

I worked on an indoor tracking system several years ago that used custom hardware. Since then, I've seen a lot of crappy solutions that use BLE RSSI measurements. This is simply not an easy problem to solve with crude methods.

they're fingerprinting locations based on rssi from devices in the signal environment. meaning your phone tries to connect, gets a set of rssis back and that then finger prints where you're standing with the phone (by the collection of rssis). the "space" it's locating in isn't physical space but "rssi tuple" space.

So, how does it get mapped to physical space then?

As long as I am in "rssi tuple" space, I maybe know if I am in a place where I have been before, and I can learn the topology of the place in terms of "which places are adjacent".

But I can't put my location on a map. How do they do the mapping?

>So, how does it get mapped to physical space then?

don't quote me but i don't think it does. you basically go around and set up hotspots in rssi space that the server knows about. when other people enter that hotspot things happen. this isn't as silly as it sounds because for a lot of applications you don't need to know where you are in physical space but just what you're near (i.e. i put my tv somewhere, hotspot it, then when my phone gets near that hotspot it's near the tv).

Yeah, that makes sense. But that's not going to be very accurate. That will be accurate at the tuple space, but not down to the nearest centimeter in X Y coordinates or anything like that.

Plus, if there is any mass between the marker and the tracked device, that will throw off readings. RSSI measurements are really only good for open air measurements or if you have characterized all the materials the rf passes through.

>lus, if there is any mass between the marker and the tracked device

there are no markers so there is no attenuation happening in that way.

Yeah, BLE beacons aren't particularly accurate for an IPS. I worked on this problem a few years ago at FB, for the purposes of helping employees find themselves + meeting rooms on a mobile app. We really only needed 2-3 meter accuracy, so people could figure out approximately where they are. BLE signals are crazy noisy, so you need to tack on a lot of filtering (kalman filter, etc) and it really helps if you have multiple data sources (wifi signals, gyroscope, RF, etc). We could also kinda cheat by adding a track to the center of each hallway on the map, then magnet users towards the track.

That sounds very similar to this! Did FB ever open-source it?

Unfortunately no. Last I worked on it it was pretty heavily integrated with their mobile maps application (https://code.facebook.com/posts/504406169737284/instrumentin...) and there wasn't any plan on open sourcing.

There was a lot of interest in BLE beacons a couple of years back for retail displays and the like. But as you say beacons don’t really work all that well, people had to install apps, etc. and the whole space seems to have sort of fizzled. The wireless beacons also don’t really have good enough battery life for a lot of applications. I played around with some of this stuff but it wasn’t good enough for anything practical that I was interested in.

I wish there was a way to do better phone tracking but it’s hard to come up with a system that would be fast and reliable if you left your phone down somewhere while not driving you crazy with false alarms on a day to day basis.

> That will be accurate at the tuple space, but not down to the nearest centimeter

Yes, you're right. Sorry, I didn't mean to imply centimeter-level precision anywhere. I said "high precision" because I compare it to GPS and not RTT (which was not common when I started this project). Unlike FIND, any nearest-centimeter precision requires specialized hardware (to be available on Android P in the future! [1]).

The FIND system fills a niche where you might need room-level or sub-room level precision without having to install anything except an app on your phone or computer.

> any mass between the marker and the tracked device will throw off readings

Its true that individual readings will be affected by varying obstacles (doors/people), and this will throw off location classification if you have very few signal generating devices in the vicinity. However, generally places will see Bluetooth/WiFi coming from their neighbors in all directions so it would be hard to attenuate all signals simultaneously. The machine learning is pretty robust too, so if one of the readings get thrown off because there are still several others that can compensate.

[1]: https://developer.android.com/preview/features.html#rtt

  nearest-centimeter precision requires specialized hardware
  (to be available on Android P in the future! [1]).

Sadly, that link says "The result is typically accurate within 1 to 2 meters" so it's nearest meter rather than nearest centimetre.

This is kind of the resolution you can get with fingerprinting: https://youtu.be/ZtnYH53A4zE

But it all depends on the density of nodes. If you are also allowed to use odometry on the phone you can suddenly go up in resolution quite a bit. In that case you'll have a conventional SLAM problem from robotics, which can be considered more or less solved.

WIFI RSSI on its own is incredibly inaccurate. In our research (previous company) we found it performed decently well in an environment with little to no other devices and simple materials. Once you add into the mix industrial material, other devices, and signals the accuracy fluctuates tremendously.

We started looking into triangulation, but didn’t have a chance to compete the project.

Is there a more product-orientated page describing what this system is actually capable of?

https://www.internalpositioning.com/doc/overview.md

Pardon the laymen’s questions, but my understanding of precision location is that it requires triangulation + accurate timestamps (the more accurate the more precise), plus with indoor the additional complexity of resolving signal multi path. There’s no mention of the hardware or “system” requirements. Possibly all modern smartphones and laptops have the necessary hardware? It would still be nice for this to mention a high level summary of the system required to set this up. How many devices to coordinate the location discovery?

The timing is less important if some of the transmitters or receivers are fixed.

GPS satellites are ambling along at 14,000 kph so they need pretty accurate timekeeping to get a precise position.

> ...it requires triangulation + accurate timestamps...

This system does not do round-trip time measurements (RTT) which I believe you are referring. RTT is different because it uses actual triangulation, whereas this system merely classifies a given point in space based on surrounding electromagnetic waves (WiFi/Bluetooth/etc.). RTT is superior, what I would call extreme precision, because it is capable of cm-level precision. (I say high-precision as compared to GPS which I tried to do this with before years ago).

I love RTT, except that it is not a solution that is widely available yet. This FIND system is nice because anyone can use on their phone right now, and you don't know need any specialized hardware or need to buy any equipment. And the signal generators are already there (your WiFi, your neighbors WiFi, Bluetooth devices, etc.).

> How many devices to coordinate the location discovery?

In the active mode (where you do the scanning) it will use every device it sees in the vicinity. In a place like an apartment in the city, this would include all WiFi+Bluetooth points in your home and your neighbors, which could be dozens.

> There's no mention of hardware or system requirements.

Thanks, I'll fix this. It really depends on how you use FIND - in active mode you only need a computer or a Android smartphone. For passive mode you need some small computers and WiFi cards.

Thanks! Much appreciated response. +1

If you're connected to an AP via Wifi, can't you ping the router? That would give you a single RTT data point, which you could combine with the non-RTT measurements to enhance the precision. The router is often either the AP itself, or linked to it through a wired connection.

Thinking more about it, you could even use an ARP ping (like the "arping" tool) to try to ping the AP directly, even if it's not the router.

Yes you can do that but that runs up against the problem of "processing time" in the router. A router might process a simple echo request for +/- 1 millisecond. Since 1 nanosecond = 1 foot, this variability already makes the distance measurement meaningless for indoor positioning. The way to do it accurately requires using something like 802.11mc [1] which can vary accurately measure time using frame collisions.

[1] http://ieeexplore.ieee.org/document/6805649/

Really cool idea, and it works really well. Was able to do very precise positioning in my house, even within a single room if given multiple references. Ultimately it drained the phone battery too much to be of actual use for me in my smarthome.

Can you provide some more details about the battery drainage? Namely, what kind of phone, how much battery usage you noticed? I've tested things with my Pixel2 and I get about 1-2%/hour battery usage.

Every URL I am trying on https://cloud.internalpositioning.com is reporting 502 bad gateway. Is the hosted service currently down?

It was, should be back up now.

I wonder if the NSA has access to this kind of data at all. Theoretically if you could put a backdoor in iOS or Android you could create a nation-wide Wifi map to track people/devices with.

Google (and probably other companies) already uses WiFi data to supplement GPS: https://support.google.com/accounts/answer/6179507?hl=en

> To estimate location, it uses sources like Wi-Fi, mobile networks, and sensors.

Just check out https://developers.google.com/maps/documentation/geolocation...

Feeding it with BSSIDs and RSSI, it gives you a precise response with GPS coordinates.

There's also Mozilla Location Service – https://location.services.mozilla.com/

> […] an open service which lets devices determine their location based on network infrastructure like Bluetooth beacons, cell towers and WiFi access points

Can you integrate it with bluetooth beacons?

Yes, the scanners available for Android and a computer both will scan for Bluetooth devices and use those for positioning as well.

Does it do filtering over time?

What do you mean by filtering?

pretty neat.

how robust are the fingerprints across devices i wonder and whether this has been experimented in with strong multipath effects (thereby confounding fingerprinting).

edit:

another interesting thing is that this is basically context sensitive hashing of tuples being done using ml classification techinques

> how robust are the fingerprints across devices

I use this system to track phones and two different computers in my home without a problem, even though I learned only one device. The RSSI values are pretty well normalized so it works well.

> strong multipath effects

The signals need not be normally distributed to be effective. Because this is classification it doesn't matter if the distribution of signal levels is multimodal (i.e. it is composed of multiple signal values through multiple paths), as long as the distribution of signals are different from location to location.

Reply on news.ycombinator.com