We run benchmarks against all of our endpoints, in production. That first chart that the author shows is in fact our live benchmarking data. If providers underperform, we kick them out of the routing pool. That is why we run those benchmarks. Performance

And errrr...yeah...that auto-exacto performance chart is both 100% useless, and totally unclear. We will get that fixed. But under the covers it is doing a lot of valuable work! https://openrouter.ai/docs/guides/routing/auto-exacto