the issue is, what will we use? companies are benchmaxxing, so how do you make a bench that cannot be cheated?
Human curated benches aren't accurate enough
Human curated benches aren't accurate enough
You go from one prompt to the final solution, whereas for many of us it is about the experience of iterative, multi turn working.