Hacker News Reader

Comment by thomascgalvin | Hacker News Reader

> A compiler will "correctly" process input into unintuitive output, but that's not always what the user intended

Importantly, this behavior is deterministic. If the compiler takes your code and spits out something you did not expect, it will always spit out that same result, given the same input. Once you figure out what went wrong, you've solved that problem.

Generative AI is intentionally not deterministic, and that means you're going to solve the same issues -- hallucinations, primarily -- over and over again.

View on HN

Determinism isn't the main hurdle. It would be a moderate effort to engineer a deterministic claude that would be no more useful than the current one.

The key characteristic of a compiler is that it takes a source language with a well-defined meaning to a target language with a well-defined meaning.

The reason claude fails to be a compiler is that its input language (english) can't be precisely defined. (And even if that were possible, it would very often fail to preserve the meaning when "compiling")

AI could be deterministic, only it requires sacrificing performance and nobody cares enough. Although algorithms are more predictable, and you can follow an algorithm's steps (even a complex algorithm can be broken into large sequences of steps, which may be broken further).

Determinism is the wrong thing to worry about. As you say an LLM could theoretically be deterministic.

Chaos (prompt instability) is the problem that isn’t solvable if you want to treat an LLM like a compiler by doing something like checking in the prompts and rebuilding the code each time.

In general if you change a single line of source code, a compiler will not produce a completely different program. If you change a single word in a prompt, you will get output that likely doesn’t match a single line of the original output.

Chaos is what leads to unpredictability, and there’s no solution to this on the horizon.

I think you're correct, I'll just point out that compilers are also chaotic today and it is often accepted. A small change to source program can lead to big change in performance. It's especially apparent in high-level and declarative languages - SQL or anything with GC.

With respect to performance that’s true. But I’ve probably spent thousands of hours in my career on performance issues caused by small SQL changes.

And LLM chaos is unimaginably worse. A single white space can result in a program that has zero overlap with the original.

They can guarantee that certain small changes won't create a big change in performance (provably, verified through tests, or trivially). An LLM can't.

“Generative AI is intentionally not deterministic, and that means you're going to solve the same issues -- hallucinations, primarily -- over and over again.”

I see it differently. Sure there are still some hallucinations (but less and less as we progress on), but I see the AI-is-not-deterministic characteristic as a good thing, in the same way that humans and human designers not being deterministic is a good thing. Imagine giving a project/task to a human, and it was completed exactly in the same way with exactly the same outputs as another human that the project was tasked to. Would that be better than each human having varying outcomes of the project? I think most would agree the answer is no.

Reply on news.ycombinator.com