Why the world needs OpenStreetMap (2014)
blog.emacsen.net
blog.emacsen.net
Gotta say OSM has been mostly amazing to work with, but there have been a few quirks over the years. A few times the direction on one directional streets have been wrong, and when I corrected it, using our GIS data as evidence, my correction was denied because I had to physically go take a picture of the street.
It felt a little silly being a municipality and having an OSM correction denied by some hobbyist, especially because it was blocking functionality on a piece of software used by 60.000 users.
Don’t get me wrong, I blame us for not getting a verified account on a critical piece of software, but it still made me hate community driven software for around two hours.
They have to guard against abuse, and they can't necessarily accept that you're an authority on any particular location. It's easier if they demand the same proof from everyone.
That said, demanding a photo of the location seems pretty stupid. If your goal is to vandalize the OSM database, how hard can it be to Photoshop a few bogus street signs?
More or less everywhere in the Danish public sector (and other countries' as well, I'm sure) seem solidly hooked on Google services. Even when logged in, deep down in some private interaction with tax or health or legal authorities, my every move and mouseclick is reported realtime to Google Anatytics. And for maps specifically, I know a traffic guy in Copenhagen Municipiality. I'm not sure he even understands the concept of switching to an open map provider.
I suspect a concern was copyright infringment. Is your GIS data public domain? If not, then it's copyrighted[1], so 'copying' data from that into OSM would essentially be a type of copyright infringement! What's to stop you adding all the info from your GIS data into OSM? All the road names, all the shop names, all the addresses? You see the problem?
If you go there, physically, and look at the sign, or take a photo then the source of the data in OSM is your eyes, not the GIS data. Requiring someone to go there is essentially a form of "clean room reimplementation" of the data, and hence has no copyright issue.
OSM prefers "local knowledge" so if you, personally, know that that street is one-way, because for example you're familiar with the area, then that's an acceptable, copyright free source.
[1] Before people jump in, yes the USA has a 'all map data from the federal government is free' that's not the case everywhere, and yes, under US copyright law non-creative pure facts aren't copyrightable, and hence there's no worry, but in some countries (e.g. the UK) you can copyright non-creative things which you spent effort collecting ('sweat of the brow'). OSM is erring on the side of caution and being strict. And OSM is not a place to experiment with the grey areas of international copyright law)
In these situations they have tags you can add e.g. saying "X" is not the name of the street and explaining that there's an error in the data provided by source Y. This stops later people assuming that any difference between the data means that OSM is in the wrong.
Semi-automated imports do happen, but overwriting existing data as part of an import is generally frowned upon without consulting the reality as a tie breaker.
I'm the author of the post back in 2014. The sad thing is that I'm midway through writing a new post about why I feel OSM is failing.
I have largely stepped away from the project in the last two years. I still believe strongly in OSM, and what it stands for, but I feel the project has not lived up to what the public needs.
Edit: Having thought a bit more about this I guess you’re going to post it on your your blog.....
This is really a pity.
Short answer: https://blog.emacsen.net/atom.xml
Longer answer:
I switched blogging software a few months ago and thought I'd changed everything over, but apparently I'd missed the atom/rss feeds.
Thanks for catching this!
However, it seems the appropriate headers are not set in the HTML pages. So there is no indication in the browser that your site has a feed, let alone where to find it.
I propose to add something like the following line to the <head> section of every site (or at least your main site):
<link rel="alternate" type="application/atom+xml" href="/atom.xml" title="Feed">OSM is the best freely available map service in Germany. Many (if not all?) local and national institutions use it as well as many private companies.
The mobile apps got really nice and are useful for normal users for different purposes and all that for free.
There are even apps everybody can use to fill the data gaps like: https://github.com/westnordost/StreetComplete
It needs some kind of pretty hard decadence (or corruption?) to ignore that for such a broad generalization.
If I was going to write a simple one sentence or one paragraph analysis of why the project is not living up to its expectations, it would be easy and I would be done.
Instead, I'm writing an in depth analysis of what I view as the impediments to the project;s success, and it's taking me several weeks to write because it's such a complex topic.
I need to explain not only the problems to seasoned OSMers, but also make my arguments understandable and accessible to a wider audience who isn't familiar with OSM internals.
It's also important to relaize that I'm not writing this from the same position as so many newbies writing about OSM- they go to the website and say "Oh this is ugly" or "This is confusing" then leave. I was very much part of the project for quite a few years, in a number of roles.
Based on my research, most paid mapping services get expensive quickly with any serious load, even if you're just geocoding addresses. Also, if you want to do anything interesting with the data you're probably going to run afoul of your license agreement that says you can't combine their mapping data with data from other sources.
It opens up so many possibilities for small developers to create large scale mapping services that would have been prohibitively expensive or otherwise impossible. They also make it really easy to put together a quick proof-of-concept by hosting their APIs on numerous public servers with generous usage terms and no registration.
I'm working on a project right now that wouldn't be possible without OSM, for multiple reasons, so I definitely agree with the article's premise.
Nevertheless, I have this burning question on how can we prevent people to abuse OSM by making false edits intentionally. This is an important issue as a lot of applications depend on OSM.
OpenStreetMap is in that 'early' phase: tons of edits are adding new data for fundamental things: streets, houses, addresses, land uses. And it has a barrier to entry - you have to be a tiny bit nerdy to confidently make edits. But in the not-so-distant future, it will need to transition to more and more maintenance and data quality tasks.
Personally, I think a good intermediate step will be to embed better and more automated review processes. The simplest one is in use by companies/agencies that use the data already:
1. Take a snapshot of your region. Vet it for your use case and deploy.
2. Grab a new snapshot at a later date (time to update the service). Look at the changes - is anything wrong or popping out at you? If so, fix it, then download + deploy.
If this becomes the primary means of adding data to OSM, a similar two-step process could be embedded into the contribution mode: Add data, then get the edit reviewed. Plus, there are automated tools to spot and characterize changes already (e.g. QA tiles) and they'll only get better.
https://wiki.openstreetmap.org/wiki/StreetComplete
Highly recommended for on-the-fly edits. It's available on F-Droid too. The most complicated part of it is entering your OSM account credentials (and perhaps the fact that you have to tell it to Upload changes; early versions didn't even require that). From there on, you just wander around town and you click on POIs and roads that pop up, and it asks you to fill in very specific bits of missing information. What are the opening hours of this place? What's the address of this house? What kind of roof does this house have? What style of road is this? What's the speed limit? Etc. It's got pretty icons and helpful images for what you're labeling (different types of road, different types of roofs, etc), and no confusing frills.
I had the idea for exactly this sort of thing myself for a while. Thankfully somebody did the work for me.
Yet another example of how execution is what matters, not the idea.
Serious question - what can we do to lower these barriers to entry? Friendlier community norms? Documentation / tools? Social stuff (groups, meetups, etc?)
I'd love to hear people's experiences on contributing, and your honest feedback. I'm one of the "core" members of OSM (I maintain the iD editor) and we're always looking for ways to improve the new user experience.
> Personally, I think a good intermediate step will be to embed better and more automated review processes.
I agree! For now, most review is done manually by volunteers. We're looking for ways to make this process more efficient.
If you haven't already, check out the OSMCha changeset review tool: https://osmcha.mapbox.com/
It runs some checks against every changeset as they occur, and presents a newsfeed-style list for anybody to review. It also has filters, in case you are interested only in certain geographic regions.
You can't edit while wasting time on the computer. You have to actually go to places you want to add. It makes scaling hard.
When I finally hopped on my desktop computer and did the iD tutorial, a lot of things became much easier. I learned how to add roads and buildings, and started doing so. Then I noticed that my map software (osmAnd) didn't announce my freeway exits.
To figure out how to mark freeway exits properly was terrible. I found the motorway wiki page and the motorway link page, but from those pages I didn't feel like I could tag an exit properly still. When I finally found the exit_info page things made a little more sense why they are confusing: there are conflicting ways of doing it! I started tagging freeway exits near me.
One thing that would have been helpful for me (although would probably be a bit of work to implement) would be more overview-level tools. Like "I want to..." a) add a missing building, b) tag a freeway exit, c) give addresses to these buildings, etc. Granted, some of this is already done. I suppose it's really the exit tagging that is a new example.
I also have been worrying about my changes being erroneously modified by other people, without a way to "take ownership" of an area and be notified of changes- I will take a good look at that OSMCha change set tool.
Careful not to copy from other maps! That's copyright infrigment, and against the OSM rules.
Yes there is. There are some differences, which the OSM community has written up to help Wikipedians get up to speed: https://wiki.openstreetmap.org/wiki/Welcome_to_Wikipedia_use...
tl;dr OSM is all about the original research, much stricter about copyright, etc
I'd love for you to give it another chance! A few of us do monitor the changeset discussion feed in Slack, and try to speak up when we see new users being mistreated.
I wish they had a method of automatically identifying relevant signs from images with GPS data. Sooner or later, cars and AR-enabled apps will be able to produce these as by-products of normal use.
Mapillary provides sign detections to OSM:
https://blog.mapillary.com/product/2017/02/06/towards-global...
Telenav is doing similar through their OpenStreetCam project (but no good link). I'm not entirely sure, but I think they have the goal of running the detection locally.
Mapbox recently used Bing Streetside imagery to detect turn restrictions (which are important for high quality routing):
https://www.openstreetmap.org/user/daniel-j-h/diary/43242
Mapbox and Telenav also compare their incoming position data with OSM and flag discrepancies (missing roads, turn restrictions again, stuff like that).
Some years ago, in my town some moron (probably in some half-assed attempt to get the town council to deal with its incredible lack of cycling infra-structure), managed to mark pretty much every pavement as a cycleway. Of course the effect is to present the town to potential visitors as very cycle friendly. It ain't - but the council don't care.
I submitted a number of notifications to Google via the appropriate link. Even attaching screenshots from StreetView showing the "No Cycling" signage. The only response was an automated reply. Years later Google is still showing these non-existant cycleways, so I assume that all notifications of incorrect data go straight to /dev/null.
Edit: perhaps the answer is for OSM to provide a downvote button for bad features.
Thanks OpenMapChest!
I can only guess how their editing priorities are mapped out (is there a Wiki or something?) but I'm guessing that's related to visitor numbers.
Notes are basically bug reports for OSM. See https://wiki.openstreetmap.org/wiki/Notes
They're available, but it's not so easy to get it since it's always very exhaustive. They should try to review how they make data available for developers, have different data sets, appropriate file formats, etc. They should offers different level of exhaustiveness and type of maps. The most basic should only contain roads and streets with their names.
OSM.org is great as a google maps alternative for users, but the problem lies with how OSM lets it data being available. GIS systems are always sensitive to size and performance, so it's not a simple problem.
MapBox seems to be on the rise, but I don't know what they're doing exactly, and what is free or not.
I tried using their .mbtiles format once and failed, it is a combination of zlib, sqlite and google protocol buffer. The generated C++ headers were the most horrific thing I saw in my life. There are better alternative to google pbf, I guess flatbuffers for example?
I have no idea how one would or should distribute map files properly. XML is not indexed and very very large, mainstream databases are inadequate because indexing geographic data is tricky so you cannot really select a specific zone you want.
If I was openstreetmap, I would federate XML exports, and generate country exports automatically, AND generate smaller exports for smaller country regions. I don't know if a good alternative would be to provide slightly overlapping squares, so planet.osm would be divided by 16 (or 32) squares, and then each square 16 (or 32) times again.
Open platform only thrive if distribution is smooth, and for maps it's not easy but it could be done. It's only a data problem.
EDIT:
I'm talking about exports simply because OSM.org servers might never be able to hold XML API calls for every developer who wants to use it in their apps.
"basic map" with "roads and streets" is choosing one thing. What about country names? Or city/town names? What about administrative borders? which language should be included? What about turn restrictions?
As in basic map I meant something the user can recognize easily enough, but still more compact in data. If you want an user to navigate manually, this kind of data is enough.
As far as APIs for OSM raw data go, Overpass API is the most popular open source option. The osm.org API is neither suitable nor intended for building non-editing apps.
When it comes to keeping a rendering database current, of course, most people use the minutely diffs mentioned by rmc.
https://wiki.osmfoundation.org/w/images/d/d8/DisputedTerrito...
Part of the explanation for your search is that local languages are given some preference by that search engine. Searching in the US I get the city in Texas as the first result.
Searching the Arabic فلسطين returns the country.
Likely "name" on the node link.
(assuming the display you are looking at is getting the data from OSM anyway; The default tiles on osm.org will use the "name" tag on nodes for most rendered place names, other users might combine OSM data with other data sources or whatever)
I discovered routing OSRM, geojson, proprietary data formats, ETL, and lot more on GitHub and blogs.
At some point, vague appeals to freedom and decentralization need to be backed by evidence.
Mapping companies simply consolidated, and Google began agressively mapping on its own.
1. The library is better designed use out of the box 2. The tile layer is more professional to regular users
For 1, I just need something easy to use. Leaflet is great but you still need more manual coding than google. Google's library is very intuitive. For a feature, I will barely touch after implementing, I prefer Google's: less learning the api, and gets the job done.
For 2, I know there is Mapbox, but they don't provide a tile layer similar to Google or when tiled, some buildings are simply missing. I just want a Google-styled tile layer. If someone could provide this, I will switch back.
You can use https://OpenMapTiles.org to style your own map with vector files generated from OSM data.