> "I gave it the same prompt, and it came out different"
I wrote a program in C and and gave it to gcc. Then I gave the same program to clang and I got a different result.
I guess C code isn't programming.
I wrote a program in C and and gave it to gcc. Then I gave the same program to clang and I got a different result.
I guess C code isn't programming.
This is a completely realistic scenario, given variance between compiler output based on optimization level, target architecture, and version.
Sure, LLMs are non-deterministic, but that doesn't matter if you never look at the code.
I can send specific LLM output to QA, I can’t ask QA to validate that this prompt will always produce bug free code even for future versions of the AI.
The output of the LLM is nondeterministic, meaning that the same input to the LLM will result in different output from the LLM.
That has nothing to do with weather the code itself is deterministic. If the LLM produces non-deterministic code, that's a bug, which hopefully will be caught by another sub-agent before production. But there's no reason to assume that programs created by LLMs are non-deterministic just because the LLMs themselves are. After all, humans are non-deterministic.
> I can send specific LLM output to QA, I can’t ask QA to validate that this prompt will always produce bug free code even for future versions of the AI.
This is a crazy scenario that does not correspond to how anyone uses LLMs.
That we agree it’s nonsense means we agree that using LLM prompts as a high level language is nonsense.
gcc and clang produce different assembly code, but it "does the same thing," for certain definitions of "same" and "thing."
Claude and Gemini produce different Rust code, but it "does the same thing," for certain definitions of "same" and "thing."
The issue is that the ultimate beneficiary of AI is the business owner. He's not a programmer, and he has a much looser definition of "same."
Claude and Gemini do not "do the same thing" in the same way in which Clang and GCC does the same thing with the same code (as long as certain axioms of the code holds).
The C Standard has been rigorously written to uphold certain principles such that the same code (following its axioms) will always produce the same results (under specified conditions) for any standard compliant compiler. There exists no such standard (and no axioms nor conditions to speak of) where the same is true of Claude and Gemini.
If you are interested, you can read the standard here (after purchasing access): https://www.iso.org/obp/ui/#iso:std:iso-iec:9899:ed-5:v1:en
True, but none of that is relevant to the non-programmer end user.
> You are relying on some odd definitions of "definitions", "equivalence", and "procedures"
These terms have rigorous definitions for programmers. The person making software in the future is a non-programmer and doesn't care about any of that. They care only that the LLM can produce what they asked for.
> The C Standard has been rigorously written to uphold certain principles
I know what a standard is. The point is that the standard is irrelevant if you never look at the code.
Your argument here (if I understand you correctly) is the same argument that to build a bridge you do not need to know all the laws of physics that prevents it from collapsing. The project manager of the construction team doesn’t need to know it, and certainly not the bicyclists who cross it. But the engineer who draws the blueprints needs to know it, and it matters that every detail on those blueprints are rigorously defined, such that the project manager of the construction team follows them to the letter. If the engineer does not know the laws of physics, or the project manager does not follow the blueprints to the letter, the bridge will likely collapse, killing the end user, that poor bicyclist.
I don't think I am. If you ask an LLM for a burger web site, you will get a burger web site. That's the only category that matters.
If one burger website generated uses PHP and the other is plain javascript, which completely changes the way the website has to be hosted--this category matters quite a bit, no?
It matters to you because you're a programmer, and you can't imagine how someone could create a program without being a programmer. But it doesn't really matter.
The non-technical user of the LLM won't care if the LLM generates PHP or JS code, because they don't care how it gets hosted. They'll tell the LLM to take care of it, and it will. Or more likely, the user won't even know what the word "hosting" means, they'll simply ask the LLM to make a website and publish it, and the LLM takes care of all the details.
Feels like the non-programmer is going to care a little bit about paying for 5 different hosting providers because the LLM decided to generate their burger website in PHP, JavaScript, Python, Ruby and Perl in successive iterations.
It's an implementation detail. The user doesn't care. OpenClaw can buy its own hosting if you ask it to.
> Feels like the non-programmer is going to care a little bit about paying for 5 different hosting providers because the LLM decided to generate their burger website in PHP, JavaScript, Python, Ruby and Perl in successive iterations.
There's this cool new program that the kids are using. It's called Docker. You should check it out.
How do you guarantee that the prompt "make me a burger website" results in a Docker container?
At this point, I think you are intentionally missing the point.
The non-programmer doesn't need to know about Docker, or race conditions, or memory leaks, or virtual functions. The programmer says "make me a web site" and the LLM figures it out. It will use an appropriate language and appropriate libraries. If appropriate, it will use Docker, and if not, it won't. If the non-programmer wants to change hosting, he can say so, and the LLM will change the hosting.
The level of abstraction goes up. The details that we've spent our lives thinking about are no longer relevant.
It's really not that complicated.
To maybe get out of this loop: your entire thesis is that nonfunctional requirements don't matter, which is a silly assertion. Anyone who has done any kind of software development work knows that nonfunctional requirements are important, which is why they exist in the first place.
More often, the issue with legacy code is not that you don’t know how to make a change, it’s because you don’t know if and how it will blow up after making it.
No. Put yourself in the shoes of the owner of the burger restaurant (who only heard the term "JavaScript" twice in his life and vaguely remember it's probably something related to "Java", which he heard three times) and you'll know why the answer is no.
This is like saying it doesn't matter if your pipes are iron, lead or PVC because you don't see them. They all move water and shit where they need to be, so no problem. Ignorance is bliss I guess? Plumbers are obsolete!
I already addressed this exact argument in another comment. In short: hosting is an implementation detail. The LLM can solve this problem just like any coding bug. The user can give his cousin's email to the LLM, which will solve the issue by finding better hosting, rewriting the program in another language, or using Docker.
Hosting is not a hard problem to solve, compared to other issues.
Can you please point to a currently available LLM that can do this?
My brother in Christ, please get off your condescending horse. I have written compilers. I know how they work. And also you've apparently never heard of undefined behavior.
The point is that the output is different at the assembly level, but that doesn't matter to the user. Just as output from an LLM but differ from another, but the user doesn't care.
Every day you say. I program every day, and I have never, in my 20 years of programming, on purpose written in undefined behavior. I think you may be exaggerating a bit here.
I mean, sure, some leet programmers do dabble in the undefined behavior, they may even rely on some compiler bug for some extreme edge case during code golf. Whatever. However it is not uncommon when enough programmers start relying on undefined behavior behaving in a certain way, that it later becomes a part of the standard and is therefor no longer “undefined behavior”.
Like I said in a different thread, I suspect you may be willfully ignorant about this. I suspect you actually know the difference between:
a) written instructions compiled into machine code for the machine to perform, and,
b) output of a statistical model, that may or may not include written instructions of (a).
There are a million reasons to claim (a) is not like (b), the fact that (a) is (mostly; or rather desirably) deterministic, while (b) is stochastic is only one (albeit a very good) reason.
Well, you sound like an ignorant troll who came here to insult people and start fights. Which also happens a lot on the internet.
Take your abrasive ego somewhere else. HN is not for you.