CodSpeed – integrated CI tool for performance testing
codspeed.io
codspeed.io
Unfortunately, I'm not using either atm. Last I looked at them:
- codspeed's Rust integration involved stubbing out a specific benchmark library which seems non-ideal and I've moved away from that library
- bencher required manually registering your benches before using it. I have responsibility over a large number of Rust repos and try to get rid of as many per-repo steps as possible. If I can't put it in my template (https://github.com/epage/_rust) and then merge it into each of my repos to get the benefit, I'm unlikely to use it.
While I've not worked with the codspeed developers, I have been providing feedback to the bencher developers and have hope that I'll be able to eventually use a solution.
Years ago I wanted a tool to continuously build and test some soft while measuring and saving the resources consumed by the tests to spot large variations. Couldn't find one so I built my own: https://github.com/rixed/lurch It does not collect any instrumentation though, just measures cpu/io ressources in the cgroup as a whole.