A robot that back-fills coverage in tests seems... counterproductive to me.
Mutation testing changes or simplifies code that is "dead" to the tests. That is the change doesn't affect the result of the tests.
This adds tests that lock-in behaviour of previously "dead" branches. (Or tests that expose the incorrect behaviour of those branches.)
In another way it is like snapshot testing. It recording the current behaviour of the code and that behaviour can be approved/rejected by the human reviewer.
The only place LLMs should be trusted is how we all probably originally tried them out: “Write me a poem about a space traveling teddy bear in the style of Eminem.”
Trying to use LLMs for factual, numeric, logical purposes is a fundamentally flawed endeavor. Especially from our community, I’m disappointed to see this amount of blind trust and willful ignorance and lack of real engineering discipline in treating LLMs as anywhere near trustworthy.
LLMs != AGI. Hot take, even if we did have AGI, it shouldn’t be given access / free rein over code and computing systems. I trust humans, with all our flaws, because we are limited in our time and energy, and speed.
> with all our flaws, because we are limited in our time and energy, and speed.
Something that might be relevant also is that we have skin in the game. External motivation can lead us to create intentionally correct or malicious programs. A robot’s fate is irrelevant since it has no capacity to care.
I’m sure that algorithms try to emulate this skin-in-the-game factor. But it will always just be a simulation.
I'm not disappointed anymore, because I've lowered my expectations for software engineers and the tech community in general. VC-adjacent hype and seeing the world through sci-fi fantasies trumps "engineering discipline" every time, at least in its discourse (you've got both for LLMs). There's a lot of unacknowledged ignorance of areas outside the tech bubble, and also a lot of contempt for it. Also, there's a pathological inability to think through to consequences to temper a compulsion to mindlessly "build the next thing," (especially acute with AGI, which for our sake I hope is infeasible or at least impractical).
I really strongly believe someone needs to "take the keys away" from the tech community. And I say that as a member of it.
> LLMs != AGI. Hot take, even if we did have AGI, it shouldn’t be given access / free rein over code and computing systems. I trust humans, with all our flaws, because we are limited in our time and energy, and speed.
Also, more importantly, we are humans. It's literally some of the scariest, most hopeless things to imagine our world being dominated by something else, especially something else that unreachably exceeds our capabilities (I'd count a billionaire controlling an army of AGI drones to be essentially inhuman).