LLMs are an abstraction just like machine code -> assembly -> C/JVM -> some lang -> LLMs?
At some point you stopped needing to understand the layer down because the layer you were on became so good. Yes there are always corner cases, but for the vast majority of developers/engineers out there, staying at your layer was enough to make a career out of it once your layer hit a certain maturity.
People keep trying to make that analogy, but it doesn't really work because LLMs aren't deterministic like compilers and assemblers.
They’re not reproducible, nor are they even reliable right now.
Regardless, you can make them deterministic by turning the temperature down to zero. Just nobody likes doing that for whatever reason. I guess it ruins some sort of illusion people seem to like.
But it is a requirement for them to be an "abstraction just like" assemblers or compilers, which is what was being claimed.
I work at a certain level, like Ruby code. That's what I write, debug and maintain. I don't really care about the internals of the interpreter or about the source code of Linux, because these layers are taken care of, they're reliable and they're being developed by competent people. I [think I] know what Ruby code should look like in order to remain [reasonably] fast, maintainable and reliable, and it's my job to build a product out of that kind of code. If I keep writing code like that, I know for sure that I'll be able to keep building the product, because the layers underneath are deterministic. It's like the certificate chain of trust, but with "surely these people are not idiots". And that's simply not the case with LLMs.
Not only can the LLM be a massive idiot, but also an unpredictable one. I can try to warn it, steer it, police it or review as much as you ask me to, but ultimately you're asking me to delegate my job and my responsibilities to an intermediate whose reasoning I don't understand, who has no loyalty, no sense of pride, no sense of ethics, can't be taught and can't be fired.
(Which is pretty much what determinism would get us, but in these conversations way too many people seem not to understand what determinism is, so describing it in terms of actions the developer takes might work better?)
The what is the semantic mapping between <some lang> and LLMs?
I know the semantic mapping between maching code and assembly (some light weight syntax manipulation and macros). I know the one between assembly and C (the C abstract machine, which is mostly about the stack and whatever call/ret instructions pair). I know the one between C and something like python (not so much different than the one between C and assembly in mechanism).
Please talk about how you go from A LLM prompt to a piece of code in Python and guarantee the intent remains unchanged.
Basically it turns out that code is full of incidental details and what you really want is to verify the important parts, while receiving a guarantee that the vast tail of incidentals is handled "reasonably."
This happened to a coworker of mine. Generally the response from one-shotted devs is a shrug of the shoulders and "wellp, them's the breaks! As long as it looks sensible from 10,000 feet up it's still a huge productivity win." But the devil, as they say, is in the details.
I just don't see that, not least of all because natural language is inherently ambiguous, whereas all the other rungs in your latter ("machine code -> assembly -> C/JVM -> some lang") are completely unambiguous by design. Consider "I saw the man with the binoculars". Does that mean "I used binoculars to look at the man", or "The man I looked at was holding binoculars"? This is the kind of inherent ambiguity that Lojban was invented to mitigate. Maybe some day we'll write "natural" language prompts in Lojban that can be unambiguously translated by an LLM, but that sounds a lot like just using a "some lang".
nothing prevents people from committing their prompts. I've started seeing prompts being committed into repos, or at least as part of the commit message.
In any case, if in the future there's a prompt specific language, it would be committed. I dont think we've reached there yet, but i dont doubt this is on the path to the future.
You mean, like a ... programming language? Honestly I can't tell these days what is satire and what isn't.
In other words, superintelligence often referred to as AGI might either be months away or just VC-Money induced cult-speak many fall victim to.
It doesn’t matter because the only certainty is that it’s not here now, and neither tomorrow etc.
For coding you still need to be in control if you want a good result.