Does reporting each model at its best performing reasoning effort introduce a best-of-N/multiple-comparisons bias, especially if models have different numbers of effort levels?
internally ive charted everything and am satisfied that theres no meaningful rank bias introduced. weve sliced it every which way. in fact we have not even published the best looking charts for this story to be told, because we have further publishing plans on frontiercode
tldr “trust me bro” this isnt the issue and if anything we couldve done more to increase N as tedsanders below points out