About 30% of Humanity's Last Exam chemistry/biology answers are likely wrong
futurehouse.org
futurehouse.org
I think the situation is better for math and physics, where answers are more straightforward to verify; probably even worse in the humanities. I also believe that releasing the models' answers would help verify the questions, but it has never been done (possibly in fear of even more train-on-test?)
Edit: to clarify, when I contacted orgs they helped me with the problems but I guess it wouldn't happen if the problem was in opposite direction