Hacker News Reader

Comment by yowlingcat | Hacker News Reader

Couldn't this make merge behavior of the same conflict non-deterministic? If there was an LLM equivalent of a seed for an RNG, I think that would avoid this issue but I'm not sure if there is.

View on HN

There is a way you could make it deterministic. You could basically let the LLM select the resolution edits and preserve that. Just as you could write a bash script to let an LLM handle your git merges. But then we go back to why? This seems like a really high risk thing to do, since at the point of conflict you really do want to know intent. While copilot is great on the code editing level I’m not sure I’d want 20% of my conflict resolutions to be the opposite of what I intended.

Sure, you can do it in a deterministic way just setting the temperature to zero and writing the prompt always in the same way. If T=0 (you always select top scoring token) and prompt is identical, the result is always the same.

Ahh, that's a fantastic callout. I've experimented with low temperature so far but never zero temperature. Makes sense, and I think this is definitely a technique I'll use in the future.

Thank you for the legendary work on Redis by the way! It's been an invaluable part of multiple large production systems I've built and is such a well-built tool. I'm hoping from your interest in this topic (and some of the blog posts you've written about it) that some part of the roadmap for Redis might help power LLM workloads in the future.

Thanks for the nice words! Appreciated, and good hacking!

There is one. LLMs are only non-random because they use an RNG. If you use a seeded PRNG then you get deterministic behaviour.

Reply on news.ycombinator.com