Ask HN: I got 99 problems but a flaky test is one?
Fellow developer here and I am curious how many developers struggling with flaky tests in terms of keeping the CI/CD pipeline green and not fail intermittently.
- What kind of struggles are you facing today? - How are you tackling those issues? - What tools would you like to see to help combat this?
I am not sure how companies handle flaky tests, some people at companies might just shrug their shoulders and retry the pipeline until they pass (that's what I've experienced out in the field), others assign them and investigate? What do you guys do at your company?
I am keen on developing a tool that would help developers track and debug flaky tests by using tracing while running tests which I don't think any tools do in that extent today. Also utilizing LLM's to pinpoint root cause of failure would be interesting in that context to help fixing flaky tests.