On the other hand, the whole deal of the LLM is that it does so stochastically and unpredictably.
LLMs mean software developers let go of some control of how something is built, which makes one feel uneasy because a lot of the appeal of software development is control and predictability. But this is the same process that people go through as they go from coder to lead developer or architect or project manager - letting go of control. Some thrive in their new position, having a higher overview of the job, while some really can't handle it.
In those circumstances, it's delegating control. And it's difficult to judge whether the authority you delegated is being misused if you lose touch with how to do the work itself. This comparison shouldn't be pushed too far, but it's not entirely unlike a compiler developer needing to retain the ability to understand machine code instructions.
But yeah, there's currently a wide gap between that and a stochastic LLM.
And the stohastic LLM can use those tools to check whether its work was sufficient, if not, it will try again - without human intervention. It will repeat this loop until the deterministic checks pass.
Please do provide a single example of this preposterous claim.
For functionality: https://en.wikipedia.org/wiki/Unit_testing
With robust enough test suites you can vibe code a HTML5 parser
- https://ikyle.me/blog/2025/swift-justhtml-porting-html5-pars...
- https://simonwillison.net/2025/Dec/15/porting-justhtml/
And code correctness:
- https://en.wikipedia.org/wiki/Tree-sitter_(parser_generator)
- https://en.wikipedia.org/wiki/Roslyn_(compiler)
- https://en.wikipedia.org/wiki/Lint_(software)
You can make analysers that check for deeply nested code, people calling methods in the wrong order and whatever you want to check. At work we've added multiple Roslyn analysers to our build pipeline to check for invalid/inefficient code, no human will be pinged by a PR until the tests pass. And an LLM can't claim "Job's Done" before the analysers say the code is OK.
And you don't need to make one yourself, there are tons you can just pick from:
https://en.wikipedia.org/wiki/List_of_tools_for_static_code_...
Unit tests check whether code behaves in specific ways. They certainly are useful to weed out bugs and to ensure that changes don't have unintended side effects.
> And code correctness:
These are tools to check for syntactic correctness. That is, of course, not what I meant.
You're completely off the mark here.
Syntactic correctness is more or less a solved problem, as you say. Doesn't matter if the author is a human or an LLM.
But it's disingenuous to claim that even the majority of code written in the world is so difficult algorithmically that it can't be unit-tested to a sufficient degree.