Hacker News Reader

LLMs are not suitable for mutation testing. Mutation testing needs to be fast to be useful (because you need to generate and test a lot of mutated versions); an LLM-based mutator would be extremely slow as well as error-prone.

View on HN

Set aside LLMs, why does mutation testing need to be fast? It would be fine to have mutation tests run slowly, out-of-band of the main CI pipeline. They aren't mission critical, they're smoke tests for your unit tests.

Also you only need to generate a set of mutations for any particular unit once, and then again when the test code or the code under test changes.

Because mutation testing (should) generate a lot of mutants, which you then need to run your unit tests against.

Yes but my point is it doesn't have to do that fast, as in unit test speeds.

Reply on news.ycombinator.com