OpenStreetMap is in trouble – a critical take on Bing Map Builder | Hacker News Reader

OpenStreetMap is in trouble – a critical take on Bing Map Builder

openstreetmap.org

280 pointspietervdvn70 comments

I studied geography for about eight years in university and something that drove me kinda mad is how cartographic authoring tools were more like CAD software than they were like video games (I once had a short rant over dinner with the President of ESRI Canada about how my GPU is built for rendering geometries and ArcGIS ignores its existence).

On the surface, the CAD pedigree makes sense. But… why aren’t more tools like video games? Make it super easy for anyone to start getting involved.

I remember all the gatekeeping-like discussion in “Public Participation GIS” seminars: you’ve gotta be an expert to author good data because there’s so many ways to do it wrong. Blah. No.

One of my daydreams if I get FU money is to make an incredibly user friendly set of web based analytical GIS tools. Not just the basics like buffering and queries, but the entire spatial toolbox to build data processing pipelines. I want a layperson to be able to ask “where should I buy a house if I care about these four parameters?” (and for it to not be a hidden magic purpose built portal on some realtor website)

The immediate retort from my internal pessimist is “the data isn’t available or isn’t consistent“ but I don’t buy that anymore. I remember having to go ask nicely to obtain shapefiles from my city. Decades later it’s all just online. And the community can help catalog data sources, add metadata about transforming them to standard schemas, and sharing data pipelines with the community in some sort of catalog.

I really think it’s a UX problem above all else.

The StreetComplete app for Android is a bit like a game and helps update "trivial" tagging tasks on OSM. I used it a bit while cycling around my hometown. Simply stop, see what you can tag/update, and continue cycling.

There should be more of these simple apps like that

Every Door is really good too (it's Flutter, so iOS/Android). I polished off a shopping centre near here in record time with it.

Links to mentioned apps:

https://every-door.app/

https://streetcomplete.app/

RE: Canadian GIS data. I thought it was really cool that most Canadian cities offer LIDAR scans for download by the public

gamification of things is discussed, but it's still not discussed enough (one could even argue that we could reach back to pedagogics ala piaget).

that said there's one aspect about gamification is that it often stops working when information requires deep thinking

but it might still be fixable

i'd love to make everything game like, society as a whole would benefit from it (it could even save energy)

The reason that I haven't pursued a gamification approach with https://mapcomplete.osm.be is twofold:

- I'm focusing on other usability issues (and features) first - I'm worried that "mapping for points" instead of "mapping to have a good map/help the community/map what the contributor needs" will result in lower data quality. Before implementing such a thing, this should be thoroughly studied

True, and even the idea of "points" might be shallow gamification. Creating fun incentives might (I'm now knowledgeable, I'm just a person and I know the power of fun in learning and working) come in other clothes.

Good luck :)

A root cause of the problem you've identified is that the tools are designed to offer first-class support for cartographic applications that pre-date the use of computers. Almost none of this has any relevance to modern geospatial analytics but it pervades the way the software works. Users that just want to build a modern analytic application are still forced to deal with this legacy baggage even though it provides no value to them and often makes their applications worse. The learning curve is unnecessarily steep.

Like you, I am of the opinion that a new set of GIS tools that focus solely on the needs of modern geospatial analysis users would be amazing.

Another gap is that cartographic tools have a deep assumption that geometry does not move. They don't provide support for correctly reasoning about spatiotemporal relationships, which are pretty important in modern geospatial applications.

Your last paragraph resonates strongly with me. I’ve taken my GIS skills and applied it to mobile robotics and performance often falls off a cliff when the moving elements aren’t treated very carefully.

OpenLayers and the rest do a great job these days, though. But a lot of the APIs are clearly intended for set and forget. “Why would you want to update this at 60hz?” I hear it asking me.

Isn’t that the point of Carto?

Good post. I haven't used Carto, Every Door, nor StreetComplete, but some of the consistency and non-ESRI-ness that OP is seeking seems to me to be present in Regrid fka Landgrid:

https://regrid.com/why-regrid

> where should I buy a house if I care about these four parameters?

A lot of this data exists, and is freely available.

I used to work for a startup where we tried to accurately predict the amount of sun a particular plot of land would get. You can pretty much get a 3d map of the world with 5m accuracy (or even more granular, but the datasets get ridiculously big at that point), then render a 360° view from any location in the world and simulate the sun movement. You can even get 3d models of some cities to take buildings into account (the earth model is ground only). The startup eventually failed for Covid reasons.

Satellite data is available for most of the world for things like wind speed, cloudiness, vegetation, ...

If you live in a decent country you may be able to freely get data for zoning, power lines, internet, building density, public transport...

You can build a really good dataset to evaluate a particular plot of land if you combine all this data.

Anyway, hit me up if you ever get that FU money, because I'd also like to do something like this :)

> You can pretty much get a 3d map of the world with 5m accuracy (

Related, maybe OT, but you seem a person that knows a lot more about this than I would know: more exactly I wanted to ask you if you know of a decent enough method to get the height of a building based on GoogleStreetView.

Meaning, I'm "inside" GoogleStreetView, I'm looking at a "building" from outside on the street, I know the geo coordinates of the camera that took the photo, I know the geo coordinates of the building, so that I know the distance between the photo camera and the building, I can "count" (for lack of a better word) the building's size in screen pixels (or some other unit/reference? which one?), and based on all that I'm curious if one can get the real height in meters of said building.

From what I could found around on some other forums there were people mentioning the aperture of the GoogleStreetView photo camera being part of the "equation" to compute it all, and that would be a problem because we do not know the values for the aperture. I have no idea though if all that discussion was on the subject or not.

The images need to be rectified or else, yes, the lens distortion messes up the measurements.

Really you just need a length and an angle to complete the triangle. The length is easy: measure distance from base of building to any point where you know the angle. But getting the angle from a point to the top of the building is tricky. I don’t know if any clever way to estimate that without stereo imagery.

A lot of this stuff is automatically calculated when a digital surface model (DSM) is generated from stereo pairs of images. So you might be able to just look it up in Google Earth. This is the data (probably blended with many other sources such as LiDAR and ground survey data) Google uses to extrude buildings into 3D models.

P.S. this was an assignment in 1st year for me. Estimate the height of the CN Tower from some aerial photos. I feel like I should remember more.

I took a shot at this. If you assume flat terrain between you and the base of the building, the problem can be simplified to basically the distance to the building from your current position times the tangent of the angle from the horizontal plane to the top of the building, from the definition of the tangent function[0]. To find the distance you can plug in your current coordinates and the buildings coordinates in a great circle distance calculator, like this one: [1], or just use the distance measure feature in Google maps. To find the angle you can zoom in on the top of the building in street view mode, and in the URL you'll have the pitch angle as a number between 1 and 179, ending in t [2] (so for instance 150.1t is a pitch angle of 150.1 degrees). Conveniently you'll also have the coordinates of the street view position in the same URL. To get the angle from the horizontal plane you have to subtract 90, since 0 pitch is straight down.

I tried it with a tall building (Turning Torso in Malmö), and got 199 m on my first attempt and 195 on my second. The correct height is 190.4 m, so pretty close. The first attempt was 100 m away, the second more than 1.2 km.

I then tried it with the Chrysler building from the other side of East river about 1.5 km away, and got 274 m which is close to the actual 282 m.

You can probably use more sophisticated methods that downloads the 360 image as a stereographic projection and extracts the pitch angle from there, but for something that's a close enough one off I think this works ok.

[0] https://en.wikipedia.org/wiki/Trigonometric_functions#Right-...

[1] https://www.gpsvisualizer.com/calculators

[2] https://www.trekview.org/blog/2020/decoding-google-street-vi...

> You can pretty much get a 3d map of the world with 5m accuracy

I'm having trouble locating high res DEM files. I can easily find arc second granular data (~30m accuracy at my latitude), but beyond that, it appears it's locked down for academic use or by request only. Where are you getting these freely available datasets?

I'm not a GIS person, I dabble in map stuff for personal use, so forgive me if I'm missing something obvious somewhere.

The lack of high res DEM files angers me a lot as well. 30m simply isn't enough and then there are projects like TanDEM-X where it isn't clear why they can't share the 12m x 12m data with the public if it's mostly payed through public funding.

Yes, I agree that data balkanization represents a significant hindrance for further development of openstreetmap. Fifteen or so years ago I was experimenting with importing DEM files from the US into Blender for landscaping, and finding the data was frustrating. Sometimes the data was available at a state website, other times at a federal website, sometimes also available on the website of some mapping program/project. Surveys within a given geopolitical constraint would be found on the website of whichever department provided the funding. I understanding that, even if ignoring legal realities, merging this data into a single "ground" (heh) truth would be a herculean task, scientifically. But there was no common database or delivery method. Some departments would link to FTP sites, other would provide arcgis downloads via an interactive web tool. I'd have to depend on user-maintained blog posts to even find the data. And the best data was always locked away behind secure portals that I couldn't get access to.

Nowadays openstreetmap is running into the same problem at the municipality level. In the US, if I understand correctly, publicly-funded geographic surveys are under copyright to whichever governing entity and not eligible for import into openstreetmap.

Take for example the excellent app Every Door (https://github.com/Zverik/every_door) with a focus on easily adding building information. In the US, openstreetmap has no address information. Either that or very little address information - but none in my area. So it'd be easy for me to add building nodes with house numbers but I won't be doing that. Not only is it a massive commitment, but the data is already available at openaddresses.io. The catch? It's "probably" license-compatible with openstreetmap. Before merging, all that data would have to be reviewed by a team of lawyers. That's not something a software developer can help with.

In existing threads here, a top complaint about OsmAnd is that lack of address information. Try planning a route without addresses, and without constantly switching to Google Maps to convert addresses to coordinates. Not very user friendly! Opensupermaps (https://github.com/pnoll1/osmand_map_creation) is a user-maintained merge of openstreetmaps and openaddresses.io, but it doesn't integrate with OsmAnd's data update system.

And that's how data balkanization is breaking openstreetmap, right now.

If you are in the US, the USGS compiles best available data into 3dep. It's pretty amazing when it's recent Lidar (like you can see hiking trails or where they built temporary logging railroads 100+ years ago).

That’s what I’m talking about- but it’s not available to download at sub arc-second resolutions for regular normies like me.

I guess I don't understand. It's a huge, complicated dataset, but I was able to quickly go from here to 1m datasets in my region:

https://www.usgs.gov/3d-elevation-program/about-3dep-product...

energy-generating systems will always attract money, since they produce something that sells, by definition. Unfortunately you are therefore out of luck.. there will be an endless line of new entries to do what you already did..

on the other side of the coin, is water, flooding and tides. Oceans are rising, many storms are more intense and flood events are increasing. Take your 3D knowledge and apply it to safety for floods and water events. No one wants that work, it is tedious and data-heavy. Yet, that is what is needed for many people who will not or cannot do it themselves. good luck

Is it possible to calculate what the view looks like from any arbitrary coordinate on the planet? At the very least, if there is a view of water, or of trees, or an unobstructed view of the horizon? And quantify that in some simple way?

All the data is there, the ability to quantity it in some simple way is not.

It's not possible with 100% accuracy, but you might get close. You can model the land easily, but not trees or buildings.

If you're standing on a barren rock it's easy, since there are no obstructions.

If it's a forested area (you get this info from satellite images), you probably can't see water, even though there's no land between you and it, except if you're standing on a cliff or something, so the trees don't bother you.

If it's an urban area, you probably won't be able to determine whether you can see water, since there aren't really models for buildings available.

I have thought it would be cool to send in swarms of connected drones to do more detailed maps of national parks, with images... First time I thought about it was in the petrified forest in AZ. That it would be cool to actually (virtually) see more than just the path you weren't supposed to leave. But could apply to many things... lots of cameras, sensors and pictures.

You might be interested in what they are building at http://felt.com/ they have a lot of great data sets already built in and it can handle imports of all kinds of mapping data. Not for actually updating the core data of maps, but seems like the tool to address the UX of mapping multiple types of data and presenting or exploring it.

>>One of my daydreams if I get FU money [...]

Is this a situation where an alternate path could be a non-profit foundation soliciting donations and grants to get started?

Also, aren't there a new crop of GIS startups that were funded in the most recent funding cycle, or are they also experiencing "truncated runway" syndrome?

In addition to StreetComplete which others have mentioned, one of the lightning talks at FOSDEM this year was about a project called CoffeOSM, which allows you to scan your receipt from a coffee shop and add the shop to OSM.[0] It's still just a proof of concept, but hopefully it develops into something more as it's a very cool idea IMO.

[0] https://gitlab.com/netvandal/coffeosm

Seen Felt.com? What do you think?

Woah, your vision sounds very similar to the goals of an open-source web app I started building a few years ago.

https://github.com/SustainableUrbanDesign/app

The idea was inspired by Cities Skylines but I started with a 2D approach since everything is much simpler than with 3D. The idea was to help urban planners fill sustainability gaps in their cities by showing which areas needed access to amenities like bus stops, daycare, grocery stores, and schools.

The approach, as mentioned is to overlay buffers surrounding existing amenities, dissolve those buffers, and find gaps. There can even be an interactive component to show the impact of placing an amenity in a neighborhood, like those emojis that radiate outward when placing a public service in Cities Skylines. The inverse problem can easily be accommodated with the app, helping people find desirable neighborhoods for their next home.

I'd be glad to jumpstart the open-source project if I can find any contributors to help build the project.

I'd love to work on something like this, maybe after I finish the text editor game I've been making for a few years.

My guess beyond pure historical baggage is that it is hard to produce user experiences that are simple, powerful and produce consistent data at the same time. Also, niche software will always be worse than top notch game UX.

Games work because they often gradually ramp the difficulty of the problems and the applied solutions up in a very controlled way using tutorial levels, clever world design etc. This means you learn the thing without even thinking you learn it.

It would definitly be possible to do such a thing in a GIS tool and have the user do some fictional work first until they reached a certain level of proviciency and then throw them at the real data.

But without tutorializing either you trust the user and give them powerful and simple tools and live with the broken data they produce (akin to button mashing in a fighting game) or you restrict the data that can be produced and frustrate the users because they cannot get the things done they want to get done.

In the end the quality of UX is always a money thing. A game with bad UX won't sell. A niche GIS software coded by three racoons in a trenchcoat will sell, even of the UX is from the stone age — as long as all other GIS tools are on a similar level.

This is something that I'm doing with MapComplete - certain features are 'unlocked' when a certain number of changes has been reached (e.g. when the contributor has >25 changesets, the tags of the data are shown)

Just to be clear: this is an individual user's "diary" (blog) hosted at openstreetmap.org, not in any way an official statement from the OpenStreetMap Foundation.

And I agree with the author that the design of Bing Map Builder looks problematic. The biggest problem being that it's segregating its contributors from the general OSM population in the name of "privacy."

Indeed - this is my opinion, not the opinion of "OpenStreetMap". Thanks for pointing this out!

Microsoft has historically been a fairly good ally to the OpenStreetMap community. We use Bing imagery a lot in our mapping efforts, as well as their buildings dataset, just to name two examples.

I hope that Microsoft owns up to this mistake, and pivots back to supporting our community rather than fracturing it. As the article notes, the OSM community is very sensitive about how large corporations communicate with us as they contribute.

yeah but in a strange twist, the Meta Overture project appears to have hired this guy as a major architect (corrections welcome)

https://2020.stateofthemap.org/sessions/SPRQVZ/

That's what I'm hoping as well. I specifically wrote this essay with the hope that this will be corrected.

And I don't think that big, corporate involvement is necessarily evil - but we should hold them to a higher standard and be critical of them in due time.

When you open https://www.bing.com/mapbuilder/ for the first time, this text is at the bottom of its splash modal:

> Map builder is powered by data from OpenStreetMap—a free, community-driven map of the world. By continuing, you agree to the Microsoft Services Agreement. Edits you make will be visible on maps that use OpenStreetMap data.

… but it’s presented at a font size of 7px. For many people and on many devices, 7px text might as well not even be there. And #767676-on-#ffffff makes it a little worse.

(Actually, the second last word, “OpenStreetMap”, is a link-styled button that opens the help/info panel described in the article, and due to careless styling and very weird DOM structures, that word “OpenStreetMap” is then bumped to 12px while that panel is open.)

I’m honestly not certain I’ve ever encountered such-sized text before, for any purpose, let alone important stuff like this. I probably have, given how 9px was hardly rare fifteen years ago (HN the yes-we-all-know-its-font-sizes-are-way-too-small-and-why-hasn’t-sanity-been-restored-just-yet-so-we-can-all-stop-using-it-at-at-least-120%-zoom even has 7pt, which is 9⅓px), but nothing springs to mind for even 8px.

Maybe to give some more context to the use of "Map builder" as editing Software in OSM, we can look at some statistics from osm changeset. (Disclaimer: I created the statistics and website)

Looking at the usage, there was 1 contributor in 2021 (Map Builder User [1]), 1506 contributors in 2022 and 890 contributors in January 2023. In total, 2376 contributors used Map builder. As of January 2023, it's the 24th most used tool by total contributor count. [2]

Looking at the edit count: Map builder is the 98th most used editing software with 3768 edits in 2021, 69618 in 2022, 38954 in January 2023 and a total of 112340 edits. [3] It has gained in popularity in the last year. I hope these statistics can help with the discussion.

[1] https://www.openstreetmap.org/user/Map%20builder%20user [2] https://piebro.github.io/openstreetmap-statistics/#c229 [3] https://piebro.github.io/openstreetmap-statistics/#eb30

OSM should ban this sort of impersonation/layering crap, and implement detection and banning of such accounts.

Impersonation is explicitly banned https://wiki.osmfoundation.org/wiki/Terms_of_Use If it applies to the situation at hand is naturally questionable.

Well, it seems like the first step would be straightforward: Block these Microsoft accounts from editing. You can't have people contributing without accepting the license agreement.

I would normally say the next step after that is to talk to someone at Microsoft. But it's very, very difficult not to infer malice or at least unwillingness to engage, here: Microsoft previously used a single account for all of these edits: https://www.openstreetmap.org/user_blocks/5701 That account was blocked, with well-described reasons. Now they're using per-user accounts, and haven't fixed the issues that were raised. That is not very encouraging.

Microsoft is in a difficult position here. Just taking the GDPR, they are presumably the data controller and they are passing on potentially personally identifiable information to OSM without a data processing agreement. They are vulnerable to a right to be forgotten request, which they can then not fulfill. Given Bing's global reach, I'm sure this is not the only issue. I'd be really careful trying to attribute this issue to malice on Microsoft's part.

Not really. Microsoft has inserted their own identity system into an OSM application they built. There's no problem if there's a login modal to use your OSM identity to make edits. They can even have a "let Microsoft manage my OSM account" checkbox with the relevant legalese if they want that to be slick.

But there's the problem: if you don't have an OSM account, the user needs to go sign up for one. If Microsoft creates the account for them, that's not just legalese, they might still be seen as the data controller. Or, at least it may be problematic enough to give the in-house lawyers a headache.

But that's not a problem at all! That's exactly the desired/required operation. OSM wants a very clear demarcation: this may be a Microsoft application, but you are an OSM community member (if you want to contribute).

That's perfectly encapsulated by a signup/login barrier. Any slick-ness on top of that fundamental requirement is sugar

It'd be better not to build the weird limited account on top of OSM and not capture the edits. It's not a difficult position at all.

authors work site:

https://mapcomplete.osm.be/?language=en

Unfortunately, OSM takes clear political stances and does not use internationally recognized borders when they should. That is antithetical to freedom (as in free software) and is why I would never donate to the project.

could you be more specific ?

If the Openstreetmap is a description of things that are visible, like built roads, buildings and named places.. How are disputed borders stopping the useful building of a map? Doesn't the United Nations and many Federal governments, have conventions for showing and discussing borders in dispute?

Why would disputed borders in some places, stop the (obviously) global project from continuing to map the real, physical place of Earth ?

Such as?

Not OP but I can give an example. Kosovo is recognized by 101 of 193 UN members. Something can be international recognized and still whatever option you show on a map you potentially anger half the world. https://en.wikipedia.org/wiki/International_recognition_of_K... . Somaliland isn't recognized by UN https://en.wikipedia.org/wiki/Somaliland There's 4 possible maps of Western Sahara https://en.wikipedia.org/wiki/Political_status_of_Western_Sa... Crimea is either Russian or Ukranian on some maps, for a while both in OSM

There is a policy about this: https://wiki.openstreetmap.org/wiki/Disputed_territories

That policy is not uniformly applied. In some conflicts "realities on the ground" are mapped and visible, but not in others. The actual effective policy is whatever biases and prejudices OSMs contributors have.

I think this is what happens when you crowdsource things. Parts become a bit fragmented, because many different people work on them - so in effect, the product reflects the organization's structure that builds the product. Wikipedia also has this issue, on the English Wikipedia itself, and also when you look at the relations of the different language Wikipedias. I personally love these projects despite these issues.

Exactly. Which is why internationally recognized borders is the way to go. It's way better than having sensitive issues settled by what the members with the most clout in the community think because it, more or less, just codifies Western biases since most members are from Western countries.

What exactly are "internationally recognized borders"? There are countless disputes that half the world's countries see one way and the other half see another way. The UN doesn't help on this matter either, they don't often vote on borders and how they should be reflected in geospatial data.

When the term "internationally recognized borders" is used by a government or institution, it's almost always a buzzword to assist a specific agenda or to actualize a specific geopolitical alignment. It's not a technical term.

They have explicitly violated their own OTG rules with regards to Crimea.

Then they have reverted the later decision to uphold OTG rule[1] with "no explanation".

1. https://wiki.osmfoundation.org/wiki/Working_Group_Minutes/DW...

Any discussion about borders is fraught, and no matter what decision you make many people will be unhappy. I don't think it is useful to tie that to freedom

Reply on news.ycombinator.com