The issue isn't that the results are necessarily bad from using a dataset with copyrighted exams. Using it for training data is "fine". But a copyrighted eval dataset makes it basically impossible to reproduce the results. One part of the original paper was validating the use of the MITQ dataset as a way to evaluate various models.

This is seperate from the issues related to the methodology not agreeing with conclusion that GPT4 could "get an MIT degree".

The second conclusion and methodology could have been fixed potentially.