Satellites auto detect buildings in OpenStreetMap | Hacker News Reader

Satellites auto detect buildings in OpenStreetMap

medium.com

163 pointsliotier33 comments

This has been heating up. Here's the seminal paper that kicked off the Cambrian Explosion: https://www.cs.toronto.edu/~vmnih/docs/Mnih_Volodymyr_PhD_Th...

And I saw two presentations about this at State of the Map US.

* Anand Thakker of Development Seed had a cool live demo: https://www.youtube.com/watch?v=BftllmqXSpA&list=PLqjPa29lMi...

* Facebook mapped a bunch of roads in Egypt, and has an ongoing effort. Facebook apparently didn't allow their's to be taped.

I have a similar project called DeepOSM: https://github.com/trailbehind/deeposm

There is starting to be a conversation in the OSM community about allowing robot edits into the map, I'm definitely pro robot. Google has been doing it for years. Mike Migurski touched off debate on the OSM Slack and blog comments with his post: http://mike.teczno.com/notes/openstreetmap-at-a-crossroads.h... (added this link after maxerickson's comment reminded me what touched of the debate).

http://terrapattern.com is also a related project from some CMU people.

> "There is starting to be a debate in the OSM community about allowing robot edits into the map"

What do you think about a hybrid: bot detects and recommends changes, which are then queued or tagged for human verification?

I've been looking at scuba data in the OSM dataset, and there are a number of bot-added entries, with a tag "verified": "no, please check and del this tag" (e.g. Maaya Thila dive site [1]).

[1] http://www.openstreetmap.org/node/663869880

That's the status quo.

I'm in favor of a regime where there is a benchmark for the recall and precision rates for the automated analysis, and anything that hits the benchmarks is acceptable. I don't regard this as a philosophical matter - let's accept any bot analysis that hits a certain threshold.

1) We can certainly set the threshold above the TIGER import in the US, which most people agree is positive, but very crufty.

2) It's the developing parts of the world that need automated road creation the most. The opposition seems to emanate from rich, European nations.

Robot edits are great when they add data. When modifying or removing work that humans did, I'm much more hesitant. This is probably why there's opposition from "rich, European nations": the developing world has little data that anything could go wrong with.

I'm not necessarily pro autonomous robots unless they either hit very high benchmarks, or it's purely for initial groundwork (or improving on previous bots' work). I totally see how it could be great for unmapped places where it could add grass/forest areas, houses, roads, rivers, etc. but tampering with existing data with no review queue (if not an approval queue, at least post-editing review)... I'm hesitant.

Again, I wouldn't be philosophical about "When modifying or removing work that humans did, I'm much more hesitant."

If a robotic system can demonstrate some high bar of precision in modifying human updates, then let the edits happen. If a human mapped a road, then that road changed places or perhaps changed from being two-way to one-way and a robot can detect that reliably, let the robot edit.

That said, I don't think "robots changing human edits" is where the robots shine. Most human edits will be good, unless the data later changes. I agree that the low hanging fruit is 1) mapping new features, 2) cleaning up imports like TIGER, 3) providing "lint" type tools to give human editors feedback if their new edit might be wrong.

I'm curious how the lint aspect of DeepOSM will work in areas that have less room for improvement than the areas I looked at in Delaware and Maine. In those areas, the imported TIGER data is poor enough that a road by road analysis is overkill. The DeepOSM output flagged most of the roads.

Given the capacity to run it, an interesting analysis would be an evaluation of the entire United States, maybe grouped by county or so. Then people in well mapped cities looking for a project could improve the worst county in their state, or the worst county in the US, etc.

> "Most human edits will be good, unless the data later changes"

One concern is the date of the bot's data-source/satellite imagery.

- Bot reviews satellite data, creates building. - Human notices that the building's actually been demolished, so removes it from OSM. - Bot reviews satellite data, recreates building.

When I was in Georgia (the Eurasian country, not the USA state) in 2014, I was making changes to OSM and noticed that the satellite imagery was very different to what was on the ground. I couldn't find any way to work out how old the satellite data was, but my gut feel is that there will be some places where the pace of development is much faster than the rate of updates in satellite imagery.

The TIGER import is about 10 years old and still not cleaned up.

> It's the developing parts of the world that need automated road creation the most. The opposition seems to emanate from rich, European nations.

I would guess that the majority of the OSM community is from rich, European nations. If people need open road data then why not let/make them create it themselves?

Using tools to suggest changes is great but without local knowledge we get bad edits like for example some contributions by Facebook or Mapbox who seem to think their US offices know better than the locals...

> I'm in favor of a regime where there is a benchmark for the recall and precision rates for the automated analysis, and anything that hits the benchmarks is acceptable.

I think the benchmark rate is "indistinguishable from a human", a sort of OSM Turing test, perhaps. This would build in a few important safety features:

1. Human quality, which generally (but not always) means no egregious breakage. Paying some attention to the linters, for example.

2. Human presence, which means it's possible to have a conversation about the mapper's or bot's activities with that entity, or at least people in control of it.

3. Human timescales, so that it's possible to have that conversation before it's all over.

> The opposition seems to emanate from rich, European nations.

Personally, I think the difference in approach may reflect the difference between those who want to build a map as fast as possible and those who want to build a sustainable community (which makes and maintains a map).

The push towards more bulk imports seems to emanate from the USA, with Facebook testing their import in Egypt. I wonder whether they had a discussion with the Egyptian community first, or just went ahead because they thought knew best what developing parts of the world need?

I'm really glad to see all of this going on! I did a little bit of building outlining in OSM and stopped when I realized that it would probably be automated within the next few years.

> Facebook mapped a bunch of roads in Egypt, and has an ongoing effort

Worth noting that the Facebook robomapping was of extremely poor quality (it actually managed to completely break the OSRM router, which is quite some achievement) and has now been reverted for that reason.

The fallacy in the concept we're discussing here is assuming that OpenStreetMap is being held back by the speed with which we can add infrastructure geometry (I'm specifically excluding mass doodling of buildings as that is a topic on its own, and, hey, google does just fine without buildings nearly everywhere).

Given reasonable quality aerial imagery (on which automatic detection a la Facebook depends too) the limiting factor for years in OpenStreetMap has been meta data collection (names, road attributes, POIs, addresses and so on). Humans are quite fast and good at tracing roads and the couple of minutes it takes to add a few dozens of streets is just inconsequential compared to the time and effort it takes to go there and actually survey and add the meta data.

Now despite some bloggers, that themselves haven't noticed that OpenStreetMap is very different than in 2007 (but you know xkcd 386), claiming that OSM is standing still, the OpenstreetMap community is quite welcoming to new and better ways of surveying and obtaining data. Mapillary and now OSV have made lots of aspects of meta data collection easier and faster and have found wide spread adoption in OSM including automatic sign detection and so on.

But even with such advances, gathering and entering the meta data is still by the order of magnitudes the dominating time consuming process in OpenStreetMap (and for what it is worth in any mapping activity).

Tracing buildings by hand is quite time-intensive even with stuff like the buildings plugin for JOSM. And it is a prerequisite for going out and mapping addresses. In cities metadata acquisition might be a much larger thing, but it isn't here on the countryside where I live. Tracing landuse form aerial imagery is probably 90% of what I do here.

Having buildings is definitely not a prerequisite for mapping addresses (when I go mapping I will typically and them before too, but again that is only a minor amount of time compared to actually gathering the data).

Mass mapping buildings is a rather recent phenomenon in OpenStreetMap and you can have a high quality, fully functional map with not a single building outline (as google shows too). Not to mention that we have had a country with full address coverage since ages that only has a small number of buildings mapped (DK).

Now there are certain high detail tasks for which having building outlines make sense: for example adding entrances with addresses, but that is fairly far along on the path of iteratively adding more detail to OSM.

I would think a tool that can quickly do high quality building extraction would end up displacing the efforts to add buildings to OSM.

Such a tool would have a couple of advantages; the capture dates of the imagery used would be well known and the coverage would be consistent. The projects to add buildings are often hasty and involve lots of people of varying levels of experience (and diligence and so on).

Of course, there seems to be a widespread (and incorrect) mindset that working with OSM only is somehow vastly easier than working with OSM plus some external data. It's nice that data added to OSM automatically shows up in lots of places, but it is usually quite straightforward to use other data alongside the OSM data.

This space is heating up a bit.

Facebook (!) has developed a system for mapping roads and is working towards using it to add data to OpenStreetMap:

http://forum.openstreetmap.org/viewtopic.php?id=55220

https://news.ycombinator.com/item?id=12150862 (no comments)

There's several other people/groups working on similar projects, like DeepOSM:

http://www.deeposm.org/

https://news.ycombinator.com/item?id=11808639 (some discussion)

And no shortage of feels about how all the parts need to work together:

http://mike.teczno.com/notes/openstreetmap-at-a-crossroads.h...

https://news.ycombinator.com/item?id=12187782 (no comments)

http://tomlee.wtf/2016/07/31/introductory-openstreetmap-poli...

http://www.maproomblog.com/2016/08/openstreetmap-at-the-cros...

There have been earlier efforts to do automated feature extraction, but it seems that improvements in AI and better, more liberally licensed imagery are moving things towards some sort of tipping point.

This is an interesting area and the most important recent change is the opening of the data. More and more municipalities provide web services where you can download high-quality imagery, e.g. have a look at https://www.wien.gv.at/ma41datenviewer/public/ in the left menu you can select "Luftaufnahmen" (aerial imagery) or DOM (Digitales Oberflaechenmodell == Digital Surface Model) and the resolution is quite high (unfortunately the imagery is watermark-raped, but most training methods are robust against that kind of errors). One might think aerial imagery is not a problem, as we have Google Maps, Bing, etc., but you reach there (free) API limits quite fast, if the goal is to get bulk data to train classifiers.

In particular laser data is a game changer and makes building footprint detection much simpler (you can make pretty training masks from the provided vector layers and train e.g. a NN for segmentation). Current state-of-the art in building and street detection from photos only is around 85% to 95%, which is quite high, but a bit too low for fully automatic usage in e.g. urban development or environmental engineering. In particular there are images where it is nearly impossible to distinguish the roof from the nearby place, if the roof is not enclosed by a green area or a garden.

At the moment classifications/segmentations are polished by humans (and providing accurate (satellite or aerial) imagery classification is a big business), but DSM data is so good, that it makes fully automatic approaches feasible and I would say one could reach ~99% there.

> The green buildings are existing structures captured in OpenStreetMap. The red buildings are the ones we detected as missing from the map.

And which color is existing structures not identified by the software? Because that would be a very quick and easy way to spot how well it's actually performing.

or which red buildings aren't buildings. It'd be nice to verify on a city such as Philadelphia that has an open parcel layer, but then again cities with parcel layers are probably well mapped out in OSM

Yeah, usually there are scripts that can update and import building outlines from open government data. Usually this is done with human guidance and by request. So if a mapper in a certain area notices missing or outdated building, he or she might request that someone in the community familiar with those import tools import the outlines for that area. Ideally the local mapper can then verify and enrich this data.

In a lot of developed countries almost any building barring small garden sheds is meticulously described in government parcel databases with decimetre precision. In the past few years these outlines have been imported in the Netherlands in OpenStreetMap; it really looks a lot better than just a bunch of rectangles.

Doesn't this mean the OSM repository will be used as a cache of the output of someone's version of some ml algorithm?

Why not distribute the data as well as the tagger and let the users tag the data themselves? This will give more freedom to the user and arguably lead to a better quality data (e.g. some additional tuning could give better results than the general case for a given region, which can be best performed by a team more familiar with that particual area) as way more eyeballs are now on the tagger itself, not just its output.

Here's a demo of the project: http://ryfeus.github.io/urban_osm_sentinel/demo/

Also does anyone know if it's possible to get the code for this? Haven't found a github or bitbucket repo for it yet.

As an infrequent but repeat contributor to OSM, this is fantastic! While there is an historic bias against automated edits (sometimes for good reasons but I do often wonder...), automating the tedious parts of mapping is, imho, a very good thing.

That said, I don't see much detail here regarding what happens when a missing building is detected. Does it get automatically added to the map? Or maybe a note is made indicating a building needs to be added? Maybe the building is auto-added but with a FixMe note for manual mappers to examine it for accuracy?

Automated building detection isn't exactly novel in remote sensing. People have been doing it for years so they can measure how fast cities develop, monitor inhabited areas after natural disasters and so on. This kind of work is never 100% correct. It usually relies on multi-band imagery so you can figure out what's vegetation and what's roof tile.

As far as I know Google does something similar to automate mapping of roads, and then they have a huge team of people cleaning up afterwards.

The best solution here (IMO) would be to add a list of potential buildings that should be checked by people either by cross-referencing satellite imagery or by people who actually live in the area.

Nice! OSM data unfortunately wasn't comprehensive enough for a project of mine, but I still love what they do. It's great seeing these kinds of improvements!

What was your project, and what did you end up using?

unstructured blob:

I'm working with data from a FOIA request for ~5yrs worth of parking tickets in Chicago. About 18m rows in total. The end goal is to be able to say whether a parking ticket was valid or not. It's been ongoing for about two years now, but when I have chunks of time I find myself getting back into the old code or just rewriting chunks. Part of the reason it's taking so long is that a personal requirement of mine is to treat every ticket on the same footing, so throwing rows out during cleanups etc isn't something I'm necessarily comfortable with. It's kinda frustrating but it's a hopeful goal.

Since ticketers don't record lat/long and only [badly typo'd] addresses, I'm stuck doing the address to lat/long translation myself. It's not easy by any means since there are many, many many edge cases. The rate limits on common street address resolution tends to be pretty low (~10k/day) and the paid APIs are ungodly expensive.

SM has been a great tool to help things go a bit forward, but when an address is missing, so is its lat/long which in many cases means rows get thrown out - which I'm not comfortable with. Interpolation using shapefile data would work, but the number of edge cases makes that difficult. I believe the data I have right now uses a combination of census data, OSM data and shapefiles from data.cityofchicago.org. Someone in the local data civics group put together a cleaned up copy of that where I was originally using the sources separately. I can pull it if you're curious.

To complicate things, having missing data adds noise and false positives when correcting typos. There don't really seem to be many address sources that are consistent with address types - eg, "pl" vs "plaza" vs "place". I threw together some pretty decent python code with recursive difflib to clean typos using other typo'd addresses as a path to the correct address and it does pretty well to correct much of that. For example, in a few steps 'nrragansitt' becomes 'narragansett'.

Given that you have a fixed area the tickets can be in, and you know every possible address, it seems like you could investigate string distance measures. It would solve your pl vs plaza vs place problem.

That's what using difflib does. Plaza vs pl vs place is just one example of many, though. There are about 1k different street names in chicago and about 18k different street names in the data. Many are missing street type and there are many street names which have different street types. So that tends to open up cans of worms. That, and street names that have 'street' in their names. Also, 'avenue' is a street name which screws things up left and right.

It's not a simple problem, and none of the popular machine learning libraries which use string distance for this sort of problem have been very effective.

Have you tried https://github.com/openvenues/libpostal?

Yep - it's actually been pretty invaluable in getting this project to where it is now. :) It definitely has its own flaws, but those are easy to work around.

Edit - looks like they have a similar problem from dealing with hand typed data. From their road map:

Ambiguous token classification (coming soon): e.g. "dr" => "doctor" or "drive" for an English address depending on the context. Multiclass logistic regression trained on OSM addresses, where abbreviations are discouraged, giving us many examples of fully qualified addresses on which to train

I've drawn a few houses on OSM a few years ago. Great to see projects like this that are automating this process!

In places with an active OSM community, like Europe, the community usually maps the new buildings faster than the satillite imagery is updated.

Reply on news.ycombinator.com