I was at NIPS and talked to one of the authors. I thought the paper was interesting, but I think the "you're not paying enough" critique is spot on. Humans clearly can be better at this task---you just can't give them strong incentives to cut corners on quality, which happens with a low piece-rate and a task that takes on the order of 3 ~ 4 minutes to do properly.