They got a ROC AUC score of 0.67. This means that for a randomly chosen denied request and a randomly chosen accepted request, they will give a higher score to the accepted request 67% of times.
Even so, I still dislike the use of this figure, as ROC AUC overstates a poor score: it hints at the model being correct 67% of the time.
In fact, the model is only knowledgeably correct 33% of the time, and just guessing the rest (taking its binary-choice 'score' up to 67%).
What the algorithm performs poorly at is determining whether any single arbitrary request is accepted or rejected; that's the test that would require around 80% success rate.
Can't quite recall the paper that gives the details. Think it might be this one http://www.hpl.hp.com/techreports/2003/HPL-2003-4.pdf
"Since the AUC is a portion of the area of the unit square, its value will always be between 0 and 1.0. However, because random guessing produces the diagonal line between (0, 0) and (1, 1), which has an area of 0.5, no realistic classifier should have an AUC less than 0.5."
So it appears that not 0.8 but 0.5 is the randomness threshold; therefore 0.7 is not so bad.