Using Open Street Maps and Kalman Filters to Clean Up GPS Data | Hacker News Reader

Using Open Street Maps and Kalman Filters to Clean Up GPS Data

blog.hypertrack.io

139 pointstapan_pandita23 comments

Kalman filters are very useful. There are several variants that can be implemented, depending on the available data and what you want to achieve. I found a great place to get your head around them was Roger Labbe's tutorials:

http://nbviewer.jupyter.org/github/rlabbe/Kalman-and-Bayesia...

My favourite resource on Kalman is Tucker McClure's work: http://www.anuncommonlab.com/articles/how-kalman-filters-wor...

Thanks for sharing this! Looks super useful.

Slightly related: I'm driving through West Africa now, and I loaded OSM onto my cheap-o used Garmin. [1]

I have been absolutely shocked and blown away by the accuracy and detail of the maps. In the tiniest dirt street village, with less than 100 people, it correctly maps all the roads and even all the walking trails.

Of course, in the big cities too.

I'm staggered they have this level of detail.

[1] http://garmin.openstreetmap.nl/

Totally. Have you seen OSM Analytics - it has some really visualizations on how the data of geographies has evolved over time - in terms of roads, buildings etc. This link[1] shows Tanzania, for example.

[1] http://osm-analytics.org/#/show/polygon:ivf~DvjpdAgots%40i~s...

The mapping in west africa could be from the Humanitarian OpenStreetMap Team

Sorry to be that guy, but: OpenStreetMap. One word, not pluralised.

That's ok. As you can see, we are sticklers for accuracy too.

And I wonder if the PostgresDB they're using is PostgreSQL or something else. I googled https://betalist.com/startups/postgresdb but the link to the site doesn't work.

It's just plain old postgres

> OSM Filter: This uses publicly available OpenStreetMap road data and maps our GPS traces to nearest possible roads

OSM is considered a reference for HyperTrack -- they snap to the OSM roads. But I wonder how they [OSM] might've constructed their maps given the similarly noisy GPS samples they likely would've crowdsourced from. Do they use kalman filters/more samples/higher precision GPS reference devices?

Data quality and coverage of OSM data varies widely. Their philosophy is to accept low quality data in an uncovered area and then improve it as multiple traces along the same routes come in from contributors. So my understanding is quality quickly increases as the community of contributor in the area grows.

Besides OSM data source are not limited to individual's GPS traces, they also include publicly available satellite imagery, official data released by government agencies under appropriate licenses. E.g. In the United States, OSM uses Landsat 7 satellite imagery, Prototype Global Shorelines from National Oceanic and Atmospheric Administration (NOAA). They have managed to get their hands on high quality data sources in a lot of countries.

In the US there is also pretty active aerial imaging, between state and local efforts and the National Aerial Imaging Program. The pdf linked at this page seems to have a nice discussion of it:

http://pubs.usgs.gov/fs/2009/3055/

Microsoft allow people to use the aerial imagery that's on Bing Maps to trace things into OSM. Previously Yahoo allowed their aerial imagery to be traced from.

Great question. It's a combination of gps data[1] and a lot of manual work. Mapbox for one has strong data teams running mapping initiatives[2].

[1] https://blog.openstreetmap.org/2013/04/12/bulk-gpx-track-dat... [2] https://github.com/mapbox/mapping/wiki

> oHw they [OSM] might've constructed their maps given the similarly noisy GPS samples

Mark One Eyeball.

OSM mappers manually convert the GPS traces to roads/features. You load the GPS trace into your editor, like a background layer. Then you draw in the road based on the GPS trace, and your common sense. But now-a-days aerial imagery is used a lot instead of GPS data.

I am guessing, but I don't think much of OSM's recent data is from GPS-samples. Roads don't pop into existence regularly, and when they do get built, my guess is someone adds them manually by referencing an underlying satellite layer.

This probably doesn't apply to remote/rural/country-side areas, and for trails. In those cases, it's probably the other way around.

Satellite/Areal imagery is not the most accurate thing. It might be skewed, misaligned or stitched incorrectly.

In OpenStreetMap areal imagery is a great help, but not the single source of truth. For mappers those images are a good starting point, but there are always other objects near it that can be used as a reference, so that new objects are placed relatively to it.

One single GPS trace is not very meaningful, but if you take several of them into account - optimally from different users with different devices - outliers become more visible and a smoothened path can be derived from it.

OpenStreetMap has a theoretical resolution of a centimeter, but in reality such a precision is not necessary as the devices that are used for consuming maps have a limited accuracy. So if the path is -/+ 5 m, that's fine for us.

These days most high quality areas on OSM are traced from aerial imagery where it's available. GPS is used as one of many sources for the Map.

Not sure what stack exactly they use under the hood, but we have created a sophisticated, very fast but similar algorithm implemented in Java.

Bonus: everything is open source: https://github.com/graphhopper/map-matching including web API, routing engine, easy setup (no DB involved), simple demo UI, ...

Only disadvantage I see: we currently do not support online map matching but this will surely come.

BTW: kalman filters are very similar to our approach (hidden markov model).

BTW2: Their FAQ is ridiculous: "Do you have competitors? No..." There are many companies with tracking and also open source products ...

This seems very useful, but isn't Google Maps doing similar stuff already? And if not, what are their concerns?

The Google Maps APIs are point in time. Using OSM and Kalman filters, we can continuously snap points to the road in realtime. Plus it's a lot faster doing it on our own database than calling external APIs.

It's called snap to roads (interpolated mode): https://developers.google.com/maps/documentation/roads/snap

Reply on news.ycombinator.com