Do they use private benchmarks? Because if not, it could be selectively nerfed.

I also wonder if cache could be used to throw these off as well, where it's serving un-nerfed cache results for context windows that are identical to ones they've previously had for benchmark requests.

Seems like the only way to do it well would be to have some randomness involved that couldn't be cheated on - but you'd want to do it in a way that doesn't throw out the benchmarks too much, so your results can be compared still.