Those layers are deterministic though...
Like in Asimov's Foundation or Liu Cixin's 'Taking Care of God'.
Maybe we need two tiers of technology. Human in the loop up to the point of AI, and then the AI layer which depends on the human stack.
Most humans are dumb, from a science perspective. The ability to be successful in science has an intelligence threshold of somewhere around 120 [2], with true innovation requiring much more (as history has shown).
Do you see any alternative to 'Taking Care of God', if we continue progressing?
But, for coding specifically, coding as an unfortunate consequence of working with computers. We've been increasing abstraction since their inception, so we can do the actual work with them that we're trying to implement with code as a means to implement it. Go ask and older programmer, and he'll say the same sort of "kids these days don't understand computers", because he had to wire up the logic on a breadboard and write machine code in a hex editor, where you use a javascript library without even knowing how to implement an adder!!!
I can imagine an organization that devotes itself to preservation of the history of technology... like monastic groups that copied texts endlessly. In a post-scarcity world, that would be a sensible outcome... people are free to devote themselves to art or science or debauchery as they please...
However, compilers have long suffered from inputs not being correct, so we have put a lot of effort into finding ways to validate that the input is correct relative to the output. Which turns out works just as well for LLMs, to the point that validating that the input is correct relative to the output ends up offering the same assurances for both compilers and LLMs alike. Compiler output correctness alone is not actually all that beneficial in practice because, again, it is only as useful as the correctness of the input and inputs are not trustworthy without "double entry accounting" to go along with it. Once you have "double entry accounting" in place then you already have what you need to turn LLM correctness into being binary as well.