Hacker News Reader

Comment by brumar | Hacker News Reader

Yep, they are quite bad without stockfish. You can test it with the playchess skill in my repo. Maybe 1200/1300, who knows? Still I do think that they can, with enough time, explore multiple variations where they confront their naivety to stockfish and build up a compact picture on why move Y should have been played instead of move X. That was my intuition when building this skill.

View on HN

Have you actually tested this with quiet positions, where understanding needs to be more about different strategic or positional choices? They can explain concrete tactical lines with stockfish analysis at hand, but so can an analysis board. In my experience, even with fairly deep and accurate engine analysis, they cannot do this accurately with quiet positions. I tried to build a product around this idea and I'm convinced that even if it is possible it would require hundreds of thousands - or millions - spent on post-training to even confirm that possibility.

Good point. I had to tweak a bit the system to get more positional analysis because this is something I wanted too get. I think it's still shying away on this aspect. When it does dwell on it, I feel that when the output describe what should be the plan of both camps, it's quite convincing but only a much stronger player (or me with stockfish) could really assess this.

Anyway I am with you that deep positional appraisal is very hard and we should not expect too much from this set of skills on that front.

When I was testing I just gave it positional exercises from Silman.

Thx for tips. I just saw there are some courses on chess.com, that will be nice to compare claude understanding with Silman commentary as the gold standard.

Edit: I am pushing a small attempt on this position. I fed this one as a screenshot https://www.chess.com/lessons/roots-of-positional-understand... . I forgot to tell that it was black to move but that was fine. The analysis quality seems ok. What puzzles me is that it recognized the Carlsbad structure and talked a bit the plans around the minority attack, not only the correct explanations on g6 and why it should be played. This is where we benefit from positional stuff in the training data.

Reply on news.ycombinator.com