Okay but...closed source harnesses change multiple times per week, sometimes per day.

And they also tune the inference of the models behind it.

You're using different tooling every day. Hard to benchmark.