Hacker News Reader

Comment by scuff3d | Hacker News Reader

I know why it works, to varying and unmeasurable degrees of success. Just like if I poke a bull with a sharp stick, I know it's gonna get it's attention. It might choose to run away from me in one of any number of directions, or it might decide to turn around and gore me to death. I can't answer that question with any certainty then you can.

The system is inherently non-deterministic. Just because you can guide it a bit, doesn't mean you can predict outcomes.

View on HN

> The system is inherently non-deterministic.

The system isn't randomly non-deterministic; it is statistically probabilistic.

The next-token prediction and the attention mechanism is actually a rigorous deterministic mathematical process. The variation in output comes from how we sample from that curve, and the temperature used to calibrate the model. Because the underlying probabilities are mathematically calculated, the system's behavior remains highly predictable within statistical bounds.

Yes, it's a departure from the fully deterministic systems we're used to. But that's not different than the many real world systems: weather, biology, robotics, quantum mechanics. Even the computer you're reading this right now is full of probabilistic processes, abstracted away through sigmoid-like functions that push the extremes to 0s and 1s.

A lot of words to say that for all intents and purposes... it's nondeterministic.

> Yes, it's a departure from the fully deterministic systems we're used to.

A system either produces the same output given the same input[1], or doesn't.

LLMs are nondeterministic by design. Sure, you can configure them with a zero temperature, a static seed, and so on, but they're of no use to anyone in that configuration. The nondeterminism is what gives them the illusion of "creativity", and other useful properties.

Classical computers, compilers, and programming languages are deterministic by design, even if they do contain complex logic that may affect their output in unpredictable ways. There's a world of difference.

[1]: Barring misbehavior due to malfunction, corruption or freak events of nature (cosmic rays, etc.).

Humans are nondeterministic.

So this is a moot point and a futile exercise in arguing semantics.

This is possibly the single worst argument I've seen in defense of all the LLM shenanigans. Makes me laugh every time.

But we can predict the outcomes, though. That's what we're saying, and it's true. Maybe not 100% of the time, but maybe it helps a significant amount of the time and that's what matters.

Is it engineering? Maybe not. But neither is knowing how to talk to junior developers so they're productive and don't feel bad. The engineering is at other levels.

> But we can predict the outcomes [...] Maybe not 100% of the time

So 60% of the time, it works every time.

... This fucking industry.

Again, it's called management. You're managing something unpredictable: the LLM.

This is nothing new, at all. Do you have a strategy that makes other engineers do what you want exactly 100% of the time?

Reply on news.ycombinator.com