A poor understanding of ROC curves. A concept that is surprisingly difficult to explain to people with low maths abilities and will lead to misuse. Even without the "confidence" not being a true probability. My anecdote is I made a classifier to speed up the search through a list of images. It worked great and saved lots of money. There was an auto accept (no review) for a confidence greater than 95%. After a while they get back to me and say "we have run some analysis and it is getting about 5% or the auto accept wrong, how can we make it get them all correct?". Of course set the threshold to 100%. But then it doesn't accept anything. Then give me a few years and more devs to develop the classifier to make it more powerful. Can't you just tweak it to make it better... And so it goes