I'm a fan of including benchmarks in CI pipelines. The Windows developers should have a historical record [1] of the performance degradation and be forced to look at it daily.
To be fair, that machine is pretty high-spec for NT 3.51, given that the minimum requirements are 12 MB RAM and a 386 (which tops out at 40 MHz). It'd certainly be nice to see that kind of performance in newer Windows versions, though.