ProgramBench

The interesting question; can it rebuild it from access to a web UI?

It's going to be incredible when I can point AI at a proprietary program, and tell AI that I want something like that, but with some annoyance fixed. And suddenly, I saved a ton of monthly subscription fees, and have better software, without any real effort.

Did Vals run all those benchmarks themselves? A bunch of these models don't show up on the official ProgramBench leaderboard. https://programbench.com/extended/