> They claim that instead of a real-world accuracy, they wanted to find the “max” accuracy that their classifier was statistically capable of

Yeah, I read this on the GitHub issue a week ago and couldn't believe it. Ideally, their profile(1) should allow them to quickly admit they were wrong on such a simple issue. Pursuit of truth and knowledge, etc.

(1) a young PhD from a prestigious university

> For example, I think a neural network is capable of achieving a “max” accuracy of 100%

Why reach for such powerful tools? f(x) = random(num_classes), achieves 100% "upper bound" accuracy.