Measuring AI Ability to Complete Long Tasks – METR

very big if true!

I downloaded their data from the metr public evals GitHub repo and independently re-implemented the very basic maximum likelihood analysis, which gave very similar results to what they shared.