Hacker News Reader

Comment by steadeepanda | Hacker News Reader

LLMs are probabilistic by nature so even if you're using without temperature it doesn't remove completely this fact, it would just narrow the output. However here we're aiming for an already defined set of rules on purpose, with no LLM including in the decision workflow on purpose. You can't safely rely on LLM for security, it's contradictory because of the current nature of LLMs, which is one of the issues that we have today, and that we're trying to propose a solution for. But yeah it's possible to include an LLM in the decision workflow it's just that in comes with cons that I was trying to mitigate with this solution

View on HN

I think your solution is a good idea. I was just pushing back on why it’s a good idea. Determinism isn’t the crux. The crux is that you’re using a symbolic algorithm with well-defined formal semantics.

I was trying to show that determinism is not the crux by pointing out that there are ways to get a deterministic output from an LLM. And that thought experiment shows that determinism isn’t what’s essential.

And I will disagree about merely narrowing the outputs. If I download a local model and set the temperature to zero and give it the same prompt twice, I will get the same output. Not one of several outputs in a narrow set. LLMs are functions.

Ah okayy, yeah sure you're right. I didn't mean it that way. I mean I know we can get deterministic output from LLM but the issue is that even with that LLMs are trained on large set of data that open a surface for prompt injections and other attacks, and no matter how strong your guardrails are there's still a way to inject a prompt that even if you configure for deterministic output. So where I was going for the "determinism" was that the solution I made sits outside the LLMs it has nothing to do with the internal reasoning, and since "determinism" it ensure and safe and secure action check against the defined rules.

Maybe here I should emphasize on the fact that it's external to any LLM? I don't know.

My understanding is that yes they are probabilistic by nature in that they give you a probability distribution over all tokens in the vocabulary for the next token, but if you just take the policy of using max prob for each token their output becomes deterministic right? Same output always results for a given input.

Hmmm yes and no, here the output "tends towards" deterministic (≠ deterministic), because the internal process contains noise (noise can't be nullified, otherwise LLMs wouldn't make mistakes and wouldn't produce unexpected outcomes). And the thing is that even if the output tends towards deterministic they will always be ways to do prompt injections. So maybe I should clarify by mentioning that the solution I shared/proposed is external to the AI's internal process, it's a complete separate thing from the LLMs

Reply on news.ycombinator.com