Does every Stack Exchange post need to be hijacked by yet another whine about site policy and moderation activities?
an algorithm that predicts whether (and for what reason) a question will be closed.
which raises all sorts of questions as to what the reasons are for which questions [should][1] be closed - this is a grey area, and there is much argument on SE itself over what sort of questions are considered acceptable, to the extent that they have started retroactively disabling lots of content which doesn't fit with an arbitrary set of rules about what a Q&A site should have on it. Without some moderation obviously the site would descend into chaos, but with very heavy handed and arbitrary moderation it will atrophy and the very people creating the quality content they want will leave, leaving them with moderators (robot or not), new users and trolls, and not much else.
Personally I think it's a huge mistake for SE to start banning questions on the basis of them being not constructive. All their other categories of problem questions make sense, but trying to ban questions that are too controversial or involve opinions is IMHO unwise - that's exactly the sort of question which leads to engaging content on SO, even if some of it verges on a troll. This has been the problematic area for them and has lead to them marking lots of useful posts as not constructive even though they clearly are constructive and informative, just because they fall on the wrong side of a line decided retrospectively to declare certain questions unquestions, and others valuable.
It will be very interesting to see if any of their robot moderators are useful in delineating this more problematic area of questions which are controversial or involve opinions - potentially that is every question/answer set more complex than 'what is 2 + 2?', and that line can vary dramatically depending on the moderator, and their opinions.
So I think this raises an interesting (though perennial) question about how heavy handed community moderation should be; in some ways related to those questions raised recently about quality and moderation on HN.
In the best case you'll end up with a txtspeak and Proper English detector. But I knew that already.
I remember reading something about telling the difference between truthful and fake reviews.
(http://cornellsun.com/section/news/content/2011/09/23/cornel...)
There's more literature in this paper.
(http://www.cs.uic.edu/~liub/publications/WWW-2012-group-spam...)
Unfortunately, that is a post-question signal, and won't be able to be used in the context of the competition.
But seriously, Stack Exchange thrives because of its focus (which is maintained in large part by its culture of moderation, though only a small part of that is done by actual "moderators"; the community self-polices pretty well).
This contest is aimed at catching posts that would be closed before they're posted. Ideally we'll get a classifier that lets us give guidance to the askers. Better education of new/confused/troublesome users, leading to better posts, and fewer closed questions overall.
(Disclaimer: SE Inc. employee, yadda yadda yadda)
> To be clear, we’re not intending to have an algorithm automatically close posts. This is aimed at improving our quality metrics (which guide users before posting) and auto-flagging facilities (which help focus moderation attention).
I don't even know how you could read that blog post and come to that conclusion...
"In a perfect world, we'd be able to offer specific, targetted guidance for authors whose posts were likely to be shot down, before they ever showed up on the site, and without requiring as much up-front effort from our community."
Which is waaaaay before the classification line.
How is that not replacing moderators [in part] with "bots"?
The problem right now is that the user mostly has very little opportunity to improve his question, because most of the time it has already been closed, and has very little chance of getting opened again (unless the question is flagged for moderator attention).