Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Game [pdf]
vene.ro
vene.ro
If I read it correctly, that's a biggie, especially given the weak outperformance of chance.
And what happens if one combines that data with other data?
"[...]and you use this computer algorithm to predict betrayal, the computer will predict that you'll be betrayed 4 times during those 10 turns. That's a lot of false positives[...]"
It looks like they took all the betrayals they had, then matched each betrayal with a similar friendship that didn't end in betrayal, and somehow ended up with a data set that was about 52% lasting friendships and 48% betrayals. I think that "somehow" is that each friendship added a data point per season, and lasting friendships lasted longer than eventual-betrayal friendships.
Within that dataset, they got 57% accuracy on the question "will this friendship end in a betrayal?" That doesn't mean they can look at an arbitrary existing friendship and predict betrayal with 57% accuracy, because the base rates will be different. And in any case, it's important to distinguish between false positives and false negatives, which the accuracy score completely fails to do.
But they also looked at eventual-betrayal friendships, and attempted to predict when the betrayal would happen. (Or at least "will it happen this turn?") The base rate for that was 14% and they got an F1 score (https://en.wikipedia.org/wiki/F1_score) of 0.31, which I think is decent but not fantastic.