fishooter

Love the idea! Maybe a suggestion with the "bit rot" problem (getting out of sync with code). Code tours will likely cover the most important parts of the codebase, which should be well tested in large projects (and large projects are the ones that need the tours). You could link the comments to those tests, and if any of those tests are changed, you will be advised to take a look at the tour comment. These tests can be also discovered automatically, as there exist code coverage tools.

The post talks about coin flipping, or 0/1 classification. Many competitions use different scores however - multiclasses, learning to find bounding boxes of objects, etc. It is much less likely to find "good" answers on the test set by chance. I think the points in the article are important, but with this context become a non-issue, when a random answer is unlikely to be correct.