Mapbox CEO says the map calling New York City ‘Jewtropolis’ has been 100% fixed | Hacker News Reader

Mapbox CEO says the map calling New York City ‘Jewtropolis’ has been 100% fixed

techcrunch.com

63 pointsyincrash54 comments

And this is why incorporating user submitted information is a tricky strategy. On one hand, there sure are a lot more users than employees. On the other hand, bad actors are often far more motivated and diligent than good faith actors, and they will try to screw over your system for no good reason.

I've come to the conclusion that at least 1% of users are actively malicious with at least enough free time, resources, and coordination of a small full time team of professionals. I created and hosted a free self-moderated community which went well for a little while, until at least a couple people became obsessed with spamming the site with extreme gore (see the Mr. OP comment in http://old.reddit.com/r/editfight which is relatively tame compared to other things they've done) and evading every single method I had of stopping or banning them, until I had to just give up altogether and shut the site down. I feel bad for the moderators on imgur.com (where I got the majority of my users) because I read somewhere that many of them now have to go to therapy for some of the things they've seen in doing their job. Online communities and the moderation of them in the modern internet is problematic enough that I personally never want to tackle that again, not even by delegating it to someone else.

Hey! Just wanted to let you know that I thought Editfight was something special. I came by occasionally and drew something. Too bad it's gone now -- just a few days ago my 4 year old asked me if he could play the game where you draw something on the screen again. It's a bummer you had to shut it down, but I understand.

Thanks for the kind words, and glad to hear it made an impression! The code is sitting here on my computer just waiting to be deployed again, but it would have to be somewhere those demonic trolls can find. That said, they don't go to HN and I do own the domain name for another year, so maybe a subdomain is in order...

EDIT: (Apparently my HN account is rate-limited so I can't reply directly.) How about this: anyone who wants access, email me and I'll send you a link. The site is up and running. People can send the link to their friends, and it'll be a private HN-only sub-community.

Is there a way to federate to HN logins? Only open it to HN members?

Or at least invite only?

I found it!

https://hn.editfight.com

I just put the site up in a subdomain specific to HN, check my post history for the link. Everyone from HN is invited, tell your friends. It'll be a mini party lol

Reminds me of reddit r/place, first prominent artwork was Charizards penis

source: was a moderator for reddit r/place

In the long run, I can’t help but wonder if creating the internet was a mistake. We’ve effectively encouraged the absolute worst elements of society to come out and have a field day without the social pushback that had kept them in line for millennia.

>We’ve effectively encouraged the absolute worst elements of society to come out and have a field day without the social pushback that had kept them in line for millennia.

We've also effectively given everyone else access to information, communication and opportunities to expand their worldview that have been more profoundly transformational than even the printing press.

No, creating the internet was not a mistake, any more than any means of mass communication is a mistake because it can be misused as well as used wisely. Without the internet, global culture would be entirely controlled, centralized and limited by media conglomerates, digital rights management and distribution by physical media, and our means of mass communication limited to telephones. The same internet that grants freedom of expression to the "absolute worst elements of society" grants it to you and I as well.

The internet is just a network, but I believe the good it's done for humanity has far outweighed the ill.

Did you somehow miss all the bad things that came to light mostly thanks to the internet, after having a field day "for millennia" without any social pushback to keep them in line?

Like the sexual abuse in Hollywood, for one example.

Do you really prefer the "absolute worst elements of society" to remain hidden rather than come out?

Mistake? At least there is visibility. I don't know what the answer to the issue is, but I know hiding it in the corner, unobserved by the masses is not the answer. This leads to NO social push back, as there is nothing to push back about.

If you combine that, social media, and porn, how much of the internet is still 'wholesome'? I think niche forums are my favorite part of the internet left.

I genuinely think Social Media is going to get us all killed.

Porn might prevent the next generation from being born, worst case.

>Porn might prevent the next generation from being born, worst case.

You know it's possible to watch porn and have sexual relationships with other people though, right?

Worst case, not expected case.

Worst case scenarios have to at least be possible. I don't believe it's possible that internet porn can or will 'prevent the next generation from being born', so I reject it as a valid worst case.

Society and Distributed Systems are similar in the sense that interesting stuff happens at the tail, and if you want to make them behave properly, you need to focus on those 1% (or 0.1% of the tail cases).

For systems: queries of death, queries with highest serving latency, records with huge size compared to rest of the entries in DB, records with extremely high fan-in for your streaming pipeline, ...

For society, a tiny minority of: users who would vandalize online data sets, refugees who commit terrorism / crimes, citizens committing crimes, recipients cheating welfare programs, people avoiding taxes, ...

It is also the tail that imposes the maximum cost on your system / society.

> It is also the tail that imposes the maximum cost on your system / society.

This is not true for every case. Excessive focus on tail risks (and moral hazard,) in fact, can be absurdly costly.

See: medicare fraud prevention.

That is true, but at the same time, Wikipedia wouldn't be possible without it. The CEO confirmed that their absuse-detection system immediately flagged the label, but apparently some human reviewer approved it. Which shows that even if your system is perfect, humans are always the weakest link.

Yeah, it's clear that this strategy can work, but a lot of people underestimate how much energy must be spent policing for anti-social behavior.

> Wikipedia wouldn't be possible without it

OpenStreetMap wouldn't be possible without it either. :)

For OSM, humans are the strongest link, they are often the best way to fix these kinds of changes. A strong local community is the solution. OSM fixed this vandalism within 2 hours.

It continues to amaze me that OpenStreetMap is not massively plagued by this (I know that MapBox uses OSM as one of many sources, as mentioned in the article, but the changed name did not appear in the live OSM map, afaik, so I am guessing it came from somewhere else). In the past, the main explanatory argument was always that OSM's complicated editing process was the gatekeeper, but I don't think this is true anymore. With the OSM web editor, it is extremely simple to edit the map. It would only take a few clicks to immediately publish (for example) a highway in the form of a swastika to the live map, but despite using the map multiple times per day, both for private use and professionally, I have never encountered something like this.

It was spotted and fixed within 24 hours in OSM (about 30 days ago), that's the power of a community driven map.

Mapbox don't keep their maps up to date, and probably used a snapshot of the data which happened to have gotten in the few hours before it was reverted.

Do you happen to know the OSM relation ID that was affected? The "New York City" relation [1] does not have anything related in its history [2]

[1] https://www.openstreetmap.org/relation/175905

[2] https://www.openstreetmap.org/api/0.6/relation/175905/histor...

Labels like that are often from place=city nodes. Here's the node in question. Compare v60 (the vandalism) with v61 (the repair)

https://www.openstreetmap.org/node/61785451

I wouldn't be surprised if it was purged from the history. Otherwise might it imply that New York City was once given that name, historically?

No, the bad edit in question is still there. OSM only "redacts" changes when there is a copyright problem. The licence was changed in Sept 2012, so there are lots of redactions from before then from people who didn't agree to the new licence.

The OSM history of an object shouldn't really be seen as the historical status of the thing.

OSM doesn't record historical changes of that kind (as in differentiating it from any other edit) and the nature of a change in that way. Although one might be able to infer actual geographical changes from comparisons. It just concerns itself with a snapshot of "now".

It was fixed within 2 hours on OSM.

Facebook are doing some interesting things both having a team curate the OSM maps, and using ML to identify vandalism. Types of vandalism include both shapes and words. I'm not sure if these specific tools are included. From what I recall, results of vandalism are checked by humans to weed out false positives. There was a surprising number of bad edits around the world!

FB (like Mapbox) don't keep their OSM based maps up to date, instead they make sure it's safe. They think that's a very important, and have developed a lot of tools to do that. I'm not sure if Mapbox promise to have their maps safe and curated, they just stopped updating them frequently.

I think the reason is that OSM doesn't have a very large number of non-tech users that are apparently fun to piss off. If a /b/ type group isn't going to get a rise out of a decent number of people, they wouldn't consider it a funny target.

Mapbox is a product aimed at less tech-savvy users, so it's more likely to be a victim of trolls.

But a /b/ type group would love the idea of habbo-hotel style raids drawing swastikas in landscapes, perhaps at a scale that isn't immediately obvious. It makes me wonder how much subtle and hard to detect vandalism would appear if OSM became more popular (as a user facing product comparable in mind-share to Google Maps). I'm imagining wikipedia style edit wars over land ownership borders etc.

We are just finishing building out our OSM infrastructure, and this morning in the shower I was thinking how nice it is that we can fix the maps by adding new construction and new roads (we do real estate for part of our state), and contribute this directly to OSM and then update tiles from OSM.

Then I came in to read this story.

Yesterday we were trying to figure out what our update strategy would be, if we would pull daily deltas, do quarterly updates, etc... We discussed the idea of vandalism and didn't have a good answer for how to prevent it. We thought about lagging behind the latest changes and then doing a "catch up" if vandalism was detected. Really what we'd like to do is make changesets based on the age of the change, except including our most recent changes (because our reputation is great :-), but I don't even know how to start doing that.

The age of update heuristic is likely a good start; high profile vandalism like this is likely to be noticed in a reasonable amount of time. Vandalism in low profile areas is going to be harder to find though.

I'm pretty sure you can get a stream of all the changes, so you would need to do something like have one copy of the database where you processed updates in near real time, and a second (production) copy where you had a delayed replication stream; in that stream, you'd want to look ahead at the real time database to calculate your stability heuristic.

Alternatively, you could have the realtime database also populate a changed object queue -- anytime an object is changed pull it out of the middle of the queue and place it at the end again. If it makes it to the front, pull the current object data and place it in your production database.

This should help with vandalism, but you do run the risk of accepting only parts of a coordinated edit of several objects. You could consider simply stopping replication into your production database in case anything ends up ultimately unstable, but I suspect it's hard to find a point in time where everything edited before was not edited again in the next 15 days (or whatever your threshold is).

This is definitely a complex problem to solve, especially since you would want to have some escape hatches so your changes happen quickly, or you could flag upstream changes to apply quickly, or worst case, you might want to revert to an earlier version of the database.

If you can figure out how to divide the data into different regions, you might have better luck, but sometimes object ids are moved around quite a bit. Geodata is already messy, and having a multitude of editors doesn't help with consistency etc.

The vandalism was reverted in 2 hours over 20 days ago. If Mapbox had stayed current with the diffs we publish, the vandalism would have been gone in a blink.

One of the news reports I read said the Mapbox press release said their AI flags over 70,000 suspicious edits a day as needing human review. Doesn't sound like much of an AI, and also sounds like the reason they were weeks behind in getting that update in.

Looks like https://www.openstreetmap.org/changeset/61555047 may have been the source of this. It's the only changeset in the history of OSM for the area that contains the offending term. It's from August 10, and was reverted on August 11 in https://www.openstreetmap.org/changeset/61556585

Correct. The vandalism was reverted within 2 hours and a data update publish a minute or 2 later. Mapbox import the original vandalism change only today for some reason and there seems to have been a delay in importing the revert.

Correct me if I'm missing something, but what's the usecase for user submitted data on the map?

Do city names change so frequently that they need to be dynamically updated? I can see some use for it when you get to the neighbourhood level to account for local nicknames, but not full cities.

The use case is that the entire globe is both incompletely mapped and also constantly changing: having it all editable is much easier than differentiating each and every case where some element might or might not change frequently.

Such a system of differentiation of map elements by likelihood of change could be good in the long term, but it would be a huge undertaking to classify them.

Do city names change so frequently that they need to be dynamically updated?

Maybe that's the bug. It didn't occur to me until I read your comment [0] that the system could simply accept user-submitted data only above a certain level of detail. It won't eliminate the damage, but it would keep it more contained.

[0] OTOH, GIS isn't my day-to-day like it would be if I worked at, say, MapBox.

So far human moderation has been a lot cheaper than implementing a reputation system for editing.

(after all, someone has to be able to edit things like the label for New York)

For instance, there's tens of millions of objects in OSM. Step one to locking some of them is deciding which ones. That alone is a big project.

It sounds like building a system on top of OpenStreetMap that is similar to Wikipedia's "protected pages" could be worthwhile.

There are legitimate reasons to edit even the information for major world cities (most of them, for example, don't have translations into all major languages yet), but there's little reason to need those changes to be applied immediately.

It's ironic as this vandalism was spotted and reverted in less than 24 hours in OSM. So any improvements and community fixing of vandalism cannot be taken advantage of by Mapbox if they rely on a monthly snapshot.

If importing from OSM (or Wikipedia) it would seem sensible to only accept content that has remained unedited for the last X hours. If something has been staple for a couple of days, it's probably not outrageous vandalism.

You'll still need some way to deal with ongoing edit wars -- I'd imagine the mapping of Jerusalem to be prickly, for example. But at least you can tell that content is dangerous and direct your human review in that direction.

> this vandalism was spotted and reverted in less than 24 hours in OSM

Even better, less than 2 hours.

> what's the usecase for user submitted data on the map?

The entire map is user submitted! It's a wiki map.

OSM isn't finished (and never will be), so we need to keep adding to it. Even the place node for New York (which was edited here) has dozens of different tags as well as just the "place=city name=New York"

Helping label things like construction, store closures/openings and like you said local nicknames that might be hard to get data on from an official source.

They should be able to make it such that some attributes or information would require multiple approval levels before taking effect. City name changes, street names, bulding locations, addresses, etc. states of things might not require approvals, say street closed, buolding undervrenovation, etc.

Hahaha, do they have logs of who did this?

Yes, here is the changelog: https://www.openstreetmap.org/changeset/61555047?node_page=3

And here’s the block of the vandal: https://www.openstreetmap.org/user_blocks/2141

Kind of interesting to see all the other active blocks in effect and the corresponding explanatory notes.

Reply on news.ycombinator.com