I'm not old enough to have gone through the revolution that was programming languages that got increasingly more abstract and decoupled from the metal, but surely there's a lesson that can be learned from that era?
Human language is famously terrible at being unambiguous.
But that's the thing, you and the AI might be "on the same page" for one prompt, and then you aren't for the next.
> and yet judges and lawyers agree on how to interpret most of them
Um, no? Lawyers and judges frequently disagree on how laws should be interpreted. And it is often not written in a way that a layperson can easily understand.
> defined arbitration process to resolve any new ambiguities.
That is famously slow and expensive, and can resul in something very different from what the lawmakers originally intended.
LLMs eat away at all of these requirements.
The developers who knew how to write efficient low-level code found that there were no jobs for that anynmore, so today the developers who can work at that level are very few.
The same will happen with AI being the new abstraction. In another decade or two, very few people will be able to write code by hand. We'll need a rack of compute in a data center and multiple KW of power to do what we used to do on a desktop PC drawing a couple of hundred Watts.