Hacker News Reader

I would bet a lot of money that Astra can follow the rules of chess (perhaps if repeated within the context window). Also, this is a different argument than what I responded to.

View on HN

I can write you a benchmark to prove it even with a heavy handed system prompt Astra will make an illegal move during the course of the games first few moves are generally ok since it's just throwing out learned moves.

I'd genuinely like to see the results of that.

I would definitely take you up on that.

https://www.chessbench.org/

>GPT-6 Astra xHigh: 0.06% rejected moves

I wonder if I would do better as a human, maybe? Or would I happen to have one move in 1500+ that's not valid?

I could see myself messing up something at some point if the board is complicated enough and trying an illegal move, perhaps if a piece somewhere would attack my king if I moved another piece. Even through I do know the rules of chess, and I have played a few games once every so often.

Reply on news.ycombinator.com