Semi-formal human language will replace programming languages for a large part. As we are able to write specifications and build huge and complex systems based on it (as e.g. the Apollo program already in the sixties), an LLM is able to understand the intent and constraints from such a specifcation and generate a formal representation of it (e.g. using Abrial's Event-B) and eventually generate "proved code".
We have to remember why high-level programming languages and type systems came into being: they were essentially a tool to help humans make less errors and manage the complexity by modularization and other means. We could still write everything in assembler in principle, as they did in the sixties and even though were able to implement large systems which worked. But whe chose to use high-level languages because they make us faster and less error-prone.
An LLM doesn't suffer from (the same) human weaknesses. You could in principle use an LLM as a "compiler" from human language to assembler. But LLMs also make errors, so it would be a risk to trust this assembler (or C code). Generating a "safe" language like Rust instead of C or assember is a rather naive work-around for the time being. Actually it would be more prudent to let the LLM generate Ada/SPARK because it can formalize and prove a wider scope than Rust; and the code would be more trustful because the LLM agent would not just write arbitrary code, but also run the SPARK prover.
And now expand this idea and you'll end up with the formal methods like B, Z or VDM which were there for decades but only a few experts were able to use them. We can now let the LLM generate the "code" in those formal languages, and let the agent run the prover and make corrections until the resulting code (whether C, Ada or assembler) is provably correct.