Reasons that I can't take this benchmark seriously:

1. Existing brute force algorithms solve 40% of this "reasoning" and "generalization" test.

2. AGI must evidently fit on a single 16GB, decade-old GPU?

3. If ARC fails blind people, it's not a reasoning test. Reasoning is independent of visual acuity. So ARC is at best a vision processing then reasoning test. SotA model "failure" is meaningless. ("But what about the other format, JSON?" Yeah, I would love to see the human solve rate on that...)