Comment by faceplanted | Hacker News Reader

Comment by faceplanted | Hacker News Reader

Parent Full thread

I've never looked into it myself but when I was graduating someone in my graduating classes final project was a cheating detection system for chess that essentially ran your moves through a variety of different chess AI's like stockfish such and over the game built up a tally of matching moves for each system, and if you matched one too closely flagged you as cheating.

I asked the guy how he would cheat against it and his suggestion was basically finding chess engines with variable "skill" levels and varying it over the game, essentially pulling out miraculous recoveries or having a good start but then forcing your AI to try to select a win state a few turns later than optimal but using the same heuristics, so it's still practically unbeatable but drags people along a bit longer to disguise itself.

Also I don't know if this exists, but is it possible to tune an AI to a specific ELO to begin with? If so you could set it to gradually ramp up over a long time so it looks like you're steadily improving over months or years and get to the top, but I imagine that wouldn't appeal to the people already willing to cheat anyway.

Lichess’s cheat detection is open source (like everything else on the site) [0]. The training dataset isn’t public, though, and it’s practically undocumented. If you look in the `modules/game` directory, though, you’ll get an idea of what sort of data that goes into the model. Besides the engine analyses, it also looks at the time spent on each move (the `Emt` type is short for elapsed move time).

[0] https://github.com/clarkerubber/irwin

The blunders computers make and that human players make are/feel very different. I wouldn't expect to last long if I were cheating on, say, lichess.org by playing out the moves of a mix of engines at various ratings.

You could still use computer assistance which could suggest several different moves with explanations, and you could just pick one of those, maybe not necessarily the best move.

Basically computers can give good hints for you. Pretty sure this type of thing would be undetectable.

You can also just use the computer for the opening. If you don't have a certain opening, or response to a certain opening memorized, just use a computer. The difference between a memorized opening and what a computer would suggest is basically nil.

This is the only sort of cheating suggested in this thread that wouldn't be caught, but wouldn't actually be that helpful.

It's pretty helpful for the average human. Most people can't blast out 15 moves of theory correctly for every possible opening, so you'd avoid a lot of bad middle games or losses to traps.

It probably isn't as helpful to a grandmaster, but that's true of engine analysis in general

Unless you're very careful I think it should be detectable at those levels. If someone is taking the same amount of time when playing some rare and tricky opening line as they are when playing the QGD, that's pretty fishy if they don't have the level where you have a big variety memorized.

> It's pretty helpful for the average human. Most people can't blast out 15 moves of theory correctly for every possible opening, so you'd avoid a lot of bad middle games or losses to traps.

Yeah, although at that level lots of people are also getting into good positions, having no idea why it is good, and losing it.

I think the problem here is that chess engines are going to recommend 'non-human' moves at any point of the mid/endgame. And these 'non-human' moves could be the first/second/third best moves according to the engine. You'd have to have to have decent understanding of the game to realize what moves aren't going to blow your cover. I am confident over the long run anti-cheating software could pick this up in most cases. But also what are the incentives for someone to cheat at online chess over the long run? It wouldn't be that interesting for more than a couple days to get a quick thrill of beating a bunch of people in my opinion.

Explaining the moves is a very difficult problem, in a sense the "line" (series of moves) the computer recommends as best for both players is the explanation. Translating that information into, say, english, or some sort of useful graphical format, would be a pretty novel feature.

https://decodechess.com/ does that kind of thing.

An easy way to detect cheaters is to look at overall accuracy as some aggregate metric of deviation from 'optimal' (as per the engine(s) used). This could just be a average of squares of centipawns lost per move, or classifying moves like how chess.com does (brilliant/best/excellent/good/inaccuracy/mistake/blunder/missed-win) and then looking at average distributions from this.

Generally how bots play is that you have a good engine, and every so often it'll pick a random move or a lower scored move to lower the difficulty. The frequency of the random move increases as the difficulty is lowered. The engine detector would likely find that since the number of non engine moves would still be quite high.

Something like Leela trained for less time could produce a lower Elo bot. I can't imagine it will look very human for its skill level though. Like, being a 1600 while consistently playing 600 level one move blunders because it hasn't figured out how bishops move yet.

This still doesn't get you past the other common detection system: click stream

It's "Elo" (not an acronym). There is the Computer Chess Rating List (https://computerchess.org.uk/ccrl/4040/rating_list_all.html) which has the current rating, on some predefined hardware.

The easiest way to do what you're discussing is just giving less evaluation time to the engine - slowly increasing it. If you wanted it to be even better you could calibrate multiple engines to make an even smoother (but changing underlying evaluation).

Reply on news.ycombinator.com