Hacker News Reader

Hacker News

Comment by specialist | Hacker News Reader

Full thread

specialist

Idea: Replay games to detect cheating.

Create a fleet of chess programs, with typical resource constraints (CPU, RAM, time).

Have the chess programs replay recorded games.

Compare the suggested moves with the player's actual moves.

Divine some kind probability that player is cheating.

Extra credit for replaying that player's history of games thru the same fleet. Was the player consistent? Were there spikes of cheating?

Until there's a better phrase, I'll call this "parallel construction".

It'd be neat to run the fleet of chess programs during live matches.

Caveat: I know nothing about chess, so someone is probably doing "parallel construction" for cheat detection.

View on HN

This comment has strong "why does <your field> need a journal" vibes...

Online chess areas like lichess and chess.com absolutely have automated cheat detection which they consider along with with reports in automated banning of accounts. They consider not just whether the player is playing the excellent moves (something common at any level and very common at high levels), but also how a deep search is needed to evaluate that move, whether other obvious moves were available, how long was taken to play the move, if the position is common, etc. When these serial cheaters are banned, the sites even refund rating points to everyone who lost to them.

However, the issue of isolated cheating incidents is still a big deal, even if the serial obvious cheaters can be detected.

Thanks. I didn't ddg hard enough. r/chess tells me part of lichess' cheat detection is similar to my hunch.

So maybe my only novel suggestion is running lichess (or equiv) live in parallel. Something livestream viewers could also help with.

And I hope someone's replaying historical games.

"how a deep search is needed to evaluate"

Of course. Obvious now that you mention it.

I assumed different programs would make different suggestions. So overseers would want to run a few programs to see if any matched the player's gameplay, style, etc.

Probably the last time I played chess was on an IBM PC XT. The quality of misc game engines varied widely. Of course there's probably been a convergence since.

It's pretty easy to determine if a player "is* an engine, and all the moves are computer generated. Where it gets difficult is more subtle cheating.

Not really. There are chess engines that are indistinguishable and made to imitate the play style of a top tier human player.

Can you give an example? I haven't seen any that are indistinguishable from humans

Well, apparently Chess24 has an engine that will let you play against a simulated Magnus Carlsen at various stages and skill-levels of his career.

So you can set some engines not to pick perfect moves but to pick moves consistent with a particular rating or style...

I don't think Play Magnus is close to indistinguishable, it's a modified version of Stockfish on the backend, having the same rating doesn't mean you have the same style

Yup. It's absolutely possible to determine whether you play versus a strength reduced/weak engine or a human. Humans make human plans and play according to human best practices, which you can find in chess literature. Engines, however, do not make long term plans and often play very inhuman (perfectly precise, unobvious tactical) moves, that often suprises even on grandmaster level.

I'm not sure if saying engines don't make long term plans is accurate, since we can set them to plan an arbitrary number of moves ahead. I think you mean that they don't have a theory of mind, and thus don't modulate their play based on their opponents.

No, he means abstract plans. Abstract plan is not exact sequence of moves, but rather general idea of where you want to be in far future.

It could be modulated based on opponent, but does not have to be.

HIARCS or Lucas Chess are engines that have been made explicitly to play like a human. You won't be able to distinguish them, with some human play in conjunction, from a very strong human player.

Thanks, those do look like they are attempting to play like humans, but neither says how that claim was tested. How would you test the their ability to imitate humans?

By cheating using them and not getting called out anymore than a human player of the same elo.

That doesn't help you prove they're indisputable from humans, even if you admitted to using them to cheat. You said some engines are indisputable from humans, can you give me some evidence of this?

AlphaZero / LeelaChess - neural net / genetic learning based ones; although they're not really commercially available, technically anyone can download the code to LeelaChess and train it if given enough CPU power. They learn chess by repeatedly playing chess against itself and in such a way simulate the organic development that has occurred in human play as well.

That's the next generation of chess engines - AlphaZero trounced Stockfish in a match of 100 games with 28 wins and 72 draws and not a single loss. And if you decrease the power a little bit (take an earlier generation), it makes significantly more human-like moves than Stockfish.

true, but more subtle cheating is more unlikely to impact the results of a big competition.

it is. Eg, catching blunders or discovering key moves. GM matches are determined by one or two of these

Exactly. At the moment there is no way to determine if a strong player used an engine for a single move or not (without proctoring). And one perfect move in a critical position can make all the difference. Also more subtle cheaters would never boost their performance to carlsen levels, but instead increase their performance slightly (e.g. from 2550 to 2650 a +100 elo boost).

In professional cycling they do two blood draws for any competition. The first one they test now for any known substances, practices (transfusions are illegal, without a medical reason). The second one they save to re-test when new drugs come to light. A few notable professionals have had titles stripped for violations that were detected years after the event.

Re-running the games (especially to check for new techniques that aren't public at the time of the event) would be in the same spirit.

I think it's important to note that the second blood draw is to detect drugs which are currently banned, but not known how to test for.

Err, yes. In the case of chess, all computer assistance is always banned, but some algorithms may not match current patterns of behavior.

In the case of blood tests, there is a cat and mouse game of changing how the drugs are administered or other chemicals in the blood that reduce the effectiveness of the tests. Later on the presence of those other substances may be used as motivation for re-testing a group of people.

It's much simpler than that. Every chess move has an accuracy score where 100% means that it is optimal play according to the top chess engine(which can easily beat any human). Anyone with an abnormally high accuracy is a cheater. Anyone with a high accuracy simply needs to verify who they are a la twitter style.

There are special chess engines that don't play the optimal move but instead try to imitate a play style or some other constraint.

Did you think the article was stating that no one knew how to detect cheaters?

I think their strategy is called "proctoring".

No one strategy is sufficient.

Reply on news.ycombinator.com