Perhaps as a reminder, the issue is that the paper’s implementation of their 2-nearest neighbor secretly uses an oracle to break ties, which obviously inflates the accuracy compared to a real-world kNN classifier that has to choose heuristically. To be fair, this could be a weird implementation accident and not malice. But I think it does invalidate the results.
But rather than admit error, the author defends this choice, and does so using (in my opinion) dubious statistical arguments. Which leads me to believe that — at least at this point — they know they made a mistake and just won’t admit it.
They claim that instead of a real-world accuracy, they wanted to find the “max” accuracy that their classifier was statistically capable of. That is, the accuracy you get if the stars happen to align and you get the luckiest possible result. Well, not only is this creative new metric not described in the paper, it’s also not applied to the other algorithms. For example, I think a neural network is capable of achieving a “max” accuracy of 100%, if all the initial weights happen to perfectly encode both the training and test sets. But of course they just use standard training to give the numbers for those algorithms.