on another note: an entire paper written on one prompt - is this the state of research these days ?
finally: a giant group of data-entry technicians are likely entering these exceptions into the training dataset at openai.
Years ago I attended an AI meetup where the organizer would discuss the differences between reductionist and wholistic research.
This is reductionist research.
In this case, reducing the paper to one prompt is to allow for objective comparison of models.
Otherwise, without a reductionist approach, how does one perform such an experiment and ensure that they are comparing apples to apples?
IE, you're getting into areas that are analogous to Turing's theories. I don't think he came up with those theories overnight.