AI service gives Wikipedians ‘X-ray specs’ to see through bad edits | Hacker News Reader

AI service gives Wikipedians ‘X-ray specs’ to see through bad edits

blog.wikimedia.org

I can't help but wonder if this will have the side-effect that vandalism will become a more subtle subversion of the pages (or maybe there's already a number of people who do just that and this will thin out the field to make them more obvious)

Relevant XKCD https://xkcd.com/810/

Is there anything about the false positive rate of this system?

Any binary 'good or bad' classifier should have at least a couple of stats attached to it to give an indication of how reliable the classifier is.

To complete the circle, it would be nice to know how it performs according to this:

https://en.wikipedia.org/wiki/Precision_and_recall

Not really, but one can report a false positives and false negatives here: https://meta.wikimedia.org/wiki/Research_talk:Revision_scori...

Since it is activated on Indonesian Wikipedia (I helped them to extend the tool to Indonesian language), I noticed that this tool can hardly capture an obvious vandalism yet. Some other edits are hard to verify whether it is vandalism or not, even by human. I believe this is a work in progress and hope that it will improve over time. But hey, it is the first time a vandalism detection tool is deployed in Indonesian Wikipedia. :)

> Not really

That's worrisome then. You'd expect a bit more rigor before putting a bot like that into production.

Actually, we openly report a wide variety of statistics about our models. E.g. if you go to https://ores.wmflabs.org/scores/enwiki, the service lists out the models that are available for English Wikipedia along with test statistics. You can also browse our test statistics in our documentation. See https://meta.wikimedia.org/wiki/ORES/damaging#English_Wikipe...

Currently, we're optimizing for AUC since it (1) is hard to cheat -- unlike accuracy or precision/recall individually and (2) it represents the ranking problem well.

See https://en.wikipedia.org/wiki/Receiver_operating_characteris...

"[AUC] is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one"

Thank you. 75%, that's not bad but it's also not exactly 'x-ray eyes' grade yet. This is a pretty tough problem you're tackling, I can see now why it is not a bot but an augmentation device. It would be nice to track the accuracy over time, to see if the vandalizers wise up to how the bot makes the decisions to game it.

Actually our AUC is 90% and that beats the state of the art in the literature. Accuracy is easy to game. We could get 97.5% accuracy by just predicting False all of the time. We don't advise reviewers to use the prediction directly, but rather to set thresholds on the probability. 70% probability is a very conservative filter and will catch about 10% and a little bit less than half of them will actually be damaging.

So, this is useful enough to reduce the workload by 90% right now. As we increase the model's fitness, we should be able to drive that up to 95%, but I doubt we'll get much better than that.

> 70% probability is a very conservative filter and will catch about 10% and a little bit less than half of them will actually be damaging.

Right. This is really interesting. A while ago I wrote a classifier that predicted based on a number of criteria if a new member of a web community would be trouble or not. It also worked as an aid in classification and not as a stand-alone decision maker. You have a lot more data to work with so that should make the problem a bit more tractable.

Does it use other bits of knowledge about the edit prior probability of a revert for an edit by that particular user or network range for instance?

It's not actually doing reversions, right? It's just scoring edits. Wikipedia editors keep hot-lists of articles they watch edits for, or patrol lists of edits or page creations; ostensibly, all this needs to do is sort those lists of edits.

It's augmentation, not making any decisions on its own (and that's logical given the false positive rate). So it's just an aid at this point but it can probably help quite a bit with triage.

Yup, it's just scoring the revisions. Currently there is no straightforward way to feedback to the tool for learning false positives yet.

Welcome to the spammer vs machine learning arms race, Wikipedia. :-)

(Former YouTube Abuse team member)

Well, Wikipedia at least has humans working on it, who can catch issues and improve it.

It can’t be worse than YouTube/Google’s systems anyway.

I don't see that many bot-generated comments anymore on YouTube. The genuine user-generated comments are among the lowest-quality on any site that I frequent, but I suspect there isn't much that Google can do about that.

I’ve seen tons of spam, "buy viagra here", and I’ve seen hundreds of videos that have just an empty image, no sound, and a full-screen annotation (and a description) that link to another video site.

I see plenty, all following a very simple formula & mostly spoofing the original uploader. And those comments stay around for quite a while. They really need to improve the system. Not that there is much value in the YT comments, as you say, I just think it reflects badly on Google that they can't keep these comments out.

Can't spammers also use machine learning, or maybe I misunderstood something.

What I meant is that I expect a huge initial win, as this catches 80-90% of spammy edits, until the spammers figure out how to get around it. Hopefully the constant stream of human-created training data (in the form of reverted or non-reverted edits) helps with this. But undoubtedly there will need to be some continual feature engineering to stay ahead.

Yeah. We're actively working with patrollers to chase down mistakes and retrain the models. This is a system that will require continual improvement. Luckily, there's a lot of researchers working on this problem so we're drawing from a lot of brilliance beyond our little team at the Wikimedia Foundation.

As referenced in the article, what is "good" and "bad" is dependent on perspective.

Wikipedia has several perspectives just within the colloquial / perceived mission. Is it for academic integrity? For common education? For the rich and powerful? For the everyman? To record as much as possible? To cull the best knowledge from the stream?

Is it for progressive ideals, which may be opposed by the common majority? Or does it reflect a conservative viewpoint, adopting only what has proven to coalesce in society?

For who's society? What about two sides of a war? Or two equally opposing economic interests (entrenched oil vs alternative energy)?

"good" or "bad" isn't a number...

"Good" simply means "unlikely to be reverted". The objective is to save time for editors.

Maybe the better metric is usefulness to the reader. A suitable algorithm can recognize things like "N.N. smells", but so can any user. The current algorithms are very effective at dealing with "XY is gay" and "Buy herbal viagra"; you don't see much of that.

The really insidious stuff is spin doctors for companies and nations. So far, Wikipedia does a very poor job with geopolitical conflicts, and competing financial interests are not taken seriously enough either.

But the people who are supposed to define what's good for the reader are the editors. They show the system what they think is good by either reverting edits or not, so by saying "this edit is likely to be reverted", the AI is saying "based on input from all editors, this edit may not be good for the users".

Effectively, those are the same thing, unless you want to create a hard definition of "good for the users". But that's going around the editors, and taking their opinion of "good" out of the equation, which IMO is a very bad idea.

I feel sorry for the AI when it runs into the n-dash, m-dash, hyphen, and minus wars.

Here's one example: https://en.m.wikipedia.org/wiki/Wikipedia:Village_pump_(poli...

About 15,000 words and closed wit no consensus.

Here there's hundreds of thousands of words: https://en.wikipedia.org/wiki/Special:Search?search=dash&pre...

"Good" on Wikipedia means the opinion of the most powerful group of editors on the page.

I'm surprised it took this long. Wikipedia has what would seem to be an excellent and straightforwardly encoded set of training data: a constant stream of edits and reversions.

There are already bots and other tools in operation on English Wikipedia for fighting vandalism automatically and semi-automatically. They're mentioned in the blog post (Cluebot NG, Huggle, STiki).

That's not training a model to predict the quality of edits - that's training a model to predict the reactions of admins/editors :) Perhaps a better title is "AI service models Wikipedia editors"?

They also have a model to predict a quality of edit, based on the WP1.0 quality model that has been used as a guideline for some time.

https://meta.wikimedia.org/wiki/ORES/wp10

That's all the model needs to do to be valuable.

Well, suppose something told me what I would decide if I gave any matter 15 seconds of consideration. It would be an invaluable tool I might use often, for example for sorting and classifying, throwing out trash and so forth. For example, it would be a fantastic spam classifier, since even after 15 seconds there is no doubt. I could use it to get through hundreds of spam messages easily, or delegate things to appropriate departments and so forth.

But it would not give me "x-ray vision", since at the end the day it is the same thing I myself conclude if I give the matter 15 seconds of consideration. Something that I miss - a genuine mail written the way a spam might be ("hey! it was nice to meet you!" in the subject, etc), by a recent acquaintance I forgot I made - would be misclassified as spam. There is no x-ray vision involved here - it's not training on the data - it's training on me!

I feel the distinction is an important one.

It's 15 seconds of consideration you have to give to huge volumes of edits, so singling out the ones where that 15 seconds is likely to be profitable for an editor is extremely helpful.

(I have no idea how well the scoring works in practice).

Remember, the editorial tasks we're talking about here are enervating.

fair enough, I guess I don't like the "x-ray specs" analogy. Their example is a disruptive edit[1] where a URL is replaced by LLAMAS GROW ON TREES

perhaps rather than say it gives editors "x-ray specs", it would be fair to say it gives editors caffeine :)

[1] - https://wikimediablog.files.wordpress.com/2015/11/diff_64221...

In case anyone else is interested in the source code, it's here: https://github.com/wiki-ai/ores

This is very peculiar. New tool is introduced, which joins a set of existing tools (bots) which address the same problem. Some comparison of quality and robustness is expected.

Instead author gives us some insight into his/hers source of inspiration:

> A feminist inspiration

> “Please exercise extreme caution to avoid encoding racism or other biases into an AI scheme.” > Wnt (from The Signpost)

And few lines below we have this gem:

> While artificial intelligence may prove essential for solving problems at Wikipedia’s scale, algorithms that replicate subjective judgements can also dehumanize and subjugate people and obfuscate inherent biases.

Essence of not being sexist or racist is not to evaluate people basing on sex or race! Author instead accents that we should totally pay attention to subjective factors (such as sex or race) instead of objective factors (such as quality of edit).

And let me guess whos edits will be targeted with extra scrutiny.. Edits of people who don't buy feminist agenda!

(Disclosure, I'm egalitarian and MRA; While I fully support women rights, I just believe that modern feminism does not have anything to do with equal rights for anyone. This post nicely fall into this pattern.)

I agree the "feminist inspiration" part is a confusing non-sequitur. I think you're missing what they're trying to say, though. I interpreted that section as "we're making our algorithms and data open so that everyone can be sure they aren't implicitly racist or sexist".

Suppose, for example, that a user repeatedly rejected edits for anyone with a noticeably feminine username. Wikimedia wants to be sure that their algorithms don't take this person's rejects to be valid data. Again, they don't really say how they'll do this, which makes the whole section rather confusing.

It's a valid concern, just presented awkwardly.

Attempting to have processes that aren't implicity racist or sexist is a feminist practice?! If that doesn't qualify as an implicitly sexist statement I wouldn't know what would.

Yes, excising discrimination from processes is a feminist practice. That's not sexist, because feminism is about subtracting, not adding, discrimination based on gender.

I think poster is skeptical that this is the attempt [of feminism] - no need to sling mud.

It's the Wikimedia Foundation that is heavily invested in the more radical parts of feminism - they did sponsor the Ada Initiative, who in many people's opinion qualify as fringe. It was them who called open data feminist ideology. Is all of science suddenly feminist? After all it depends on exchange of data. That's plain barmy.

Alright buddy, I don't want to get in a flame war here, but I'll summarize my arguments in short.

A hits B. Hard. When B tries fighting back, A suggests the world should be a fair place, and really, he doesn't support violence at all, and there should be global peace.

C turns D's family into indentured servants. When D becomes a part of a revolution, and tries to make sure no one from C's (or D's) side can do it again, C says "I'm all for equality yo, lets not be violent at each other".

F's people are particularly unfairly attacked by different members of G groups, which are in power in their country. A lot of F's die unfairly and prematurely, and are mistreated for no reason. When F's people say "our lives matter, be kinder to us", some people who are in G's side say: "hey F, ALL lives matter."

People called G's are unfairly discriminated against, overtly and covertly, but often very openly. They are called names, and their presence in public is often made very difficult, by people called H. So when G's get together and try to get this G-rights going, some H's say: "Hey, there should be equality. You shouldn't get special priviledges. If you are systematically prejudiced against, your bad, there should be equality okay". Lets call them HRA's. You are HRA, exo762.

Your argument is in fact two arguments. First one is: "women are discriminated against". Second one is: "women are fighting back, men brought it onto themselves".

So lets go through part one, "women are oppressed", point by point.

> A hits B. Hard. ...

Violence. 70% of non-reciprocal domestic violence is committed by women against men. Article [0]. Men are primary target of every police intervention, even if it was man who reported violence committed against him by his woman partner. Lookup Duluth model. And battered men has nowhere to go - there are no shelters for male victims of DV.

> C turns D's family into indentured servants...

Slavery? What?

> ... members of G groups, which are in power in their country. ...

"Men in power". Yeah, those who are in power are mostly men. But what good comes from it for men who are not in power? Majority of homeless are men; majority of prisoners are men; majority of unemployed are men; majority of victims of DV are men. Men get longer sentences for same crimes, women get suspended sentences for things men serve time.

As for clear legal discrimination, there are multiple examples of it. Fathers' rights are a joke, draft is still for males only, selective service is for males only; men can't opt-out from parenthood, while options are available for women. As for legal discrimination of women - there is one clear-cut example and it is access to abortion.

As for your second argument. "Men had it coming, now repent for your sins". This argument is dumb beyond recognition. Even Joseph Stalin said that "The son is not responsible for the father's deeds". You seems to be not agreeing with him on only thing everyone agrees with him. And you basically place blame ON HALF OF HUMANITY, collectively. Good job!

[0] http://www.nationalpost.com/opinion/columnists/story.html?id...

If you want more sources, go here: https://www.reddit.com/r/MensRights/wiki/faq. FAQs don't bite, and I've spent enough time typing out this answer.

EDIT: word of warning, /r/MensRights is a bad place, too much vitriol. Go to /r/FeMRADebates instead.

That's lovely, thank you. Upvoted.

If a bot can detect that you "don't buy feminist agenda," then it is evaluating based on content, not on sex or race.

The two can correlate, so "based on content" isn't enough, "based objectively/justifiably on content" is.

e.g What if the basis was about the non-/usage of gender-neutral pronouns?

+1 for Egalitarianism. So, when I use the word "feminism", I'm referring to the academic field that explores power structures in social systems. These power structures go far beyond gender issues despite what the label "feminism" implies. It's this broader sense of justice/morality in social spaces that I would like to engage with and draw inspiration from.

There are two core bits of feminist theory that I draw from as inspiration in organizing our work on this system:

* Empowerment vs. Power over -- I'm a computer scientist and I've been studying quality control in Wikipedia for 10 years. That makes me very powerful in this conversation. I could use that power to browbeat Wikipedians into changing their quality control practices to what I think we should do. But on the other hand, I don't think that my perspective is absolute. A lot of Wikipedians know better than I do how they should curate the encyclopedia. So instead, I'm trying to empower Wikipedians and our community of tool developers to imagine how they'd like quality control to work. My research[1,2,3] suggests that we need reform here and I'd rather see that reform through innovation than command and control. Honestly though, I don't need to look to feminism for this insight. I came to the same conclusion after considering Wikipedia as a complex social system[4].

* The potential for systemic biases -- As other comments have pointed out, there's inevitably systemic bias in people's behavior as they evaluate good and bad edits in Wikipedia because these judgements are inherently subjective and there's a strong incentive to push back against anything that is new and unusual -- especially, things that don't follow the western scholarly traditions of reference information. Once encoded into a black box algorithm, these biases can become even more hidden and substantially more impactful. One bias that is plainly obvious at the moment is against anonymous editors. While a lot of vandalism may come from anonymous editors, so does ~25% of productive contributions to Wikipedia articles (it's my job to measure these things). We're actively working to nerf the "is anonymous" feature and find better sources of signal.

1. Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2012). The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline. American Behavioral Scientist, 0002764212469365. http://www-users.cs.umn.edu/~halfak/publications/The_Rise_an... 2. Halfaker, A., Geiger, R. S., & Terveen, L. G. (2014, April). Snuggle: Designing for efficient socialization and ideological critique. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 311-320). ACM. http://www-users.cs.umn.edu/~halfak/publications/Snuggle/hal... 3. Halfaker, A., Kittur, A., Kraut, R., & Riedl, J. (2009, October). A jury of your peers: quality, experience and ownership in Wikipedia. In Proceedings of the 5th International Symposium on Wikis and Open Collaboration (p. 15). ACM. http://www-users.cs.umn.edu/~halfak/publications/A_Jury_of_Y... 4. Halfaker, A., Kittur, A., & Riedl, J. (2011, October). Don't bite the newbies: how reverts affect the quantity and quality of Wikipedia work. In Proceedings of the 7th international symposium on wikis and open collaboration (pp. 163-172). ACM. http://www-users.cs.umn.edu/~halfak/publications/Don't_Bite_... 5. Geiger, R. S., & Halfaker, A. (2013, August). When the levee breaks: without bots, what happens to Wikipedia's quality control processes?. In Proceedings of the 9th International Symposium on Open Collaboration (p. 6). ACM. http://www-users.cs.umn.edu/~halfak/publications/When_the_Le... 6. Halfaker, A., & Riedl, J. (2012). Bots and cyborgs: Wikipedia's immune system. Computer, (3), 79-82. http://stuartgeiger.com/bots-cyborgs-halfaker.pdf

Sorry about the formatting issues. I can't seem to edit the post :/

Ironic, given that Wikipedia is one of the least female friendly sites on the net, with around 10% female contributors, who are constantly harassed. As well as the systemic bias against topics regarding women.

"Feminist Inspiration". Like straight out of South Park.

Basically I am counting this as anything remotely critical of any subject that the "good guys" have decided is flawless will be censored. https://twitter.com/keksec_org/status/672447312044650496

Maybe it's too cynical as it might just pandering to a bunch of Fratboys... PC-Fratboys.

Reply on news.ycombinator.com