Hacker News Reader

Comment by fny | Hacker News Reader

Secret: "compile" that orchestration prompt. Determinism is solved by turning prompts into code that can in turn run agents or run code or both.

Everyone misses this pattern with skills: you can just drop code alongside a SKILL.md to guarantee certain behaviors, but for some reason everyone's addicted to writing prompts. You don't even need to build a CLI. A simple skill.py with tasks does it. You can even have helpers that call `claude -p`!

View on HN

What about when the model trusts itself more than the "black box" you gave it, and hallucinates its use or non-use in favor of reimplementation? I found this video about "intelligent disobedience" interesting.

https://www.youtube.com/watch?v=Qu-00j9XuF0

Yeah, that's how I do skills. If I can make a script, I do. Everything that can be deterministic should be. https://github.com/notque/vexjoy-agent

Exactly this, I tend to work this way. I built an ingestion pipeline to pull concepts out of a novel using Qwen and push them into falkordb this way

Could you elaborate what does "compiling orchestration prompt" mean?

a guess but i think they mean take the orchestration prompt and prompt yet another llm to turn that prompt into code..?

When you get some abstraction working you concretize it in something deterministic, or sort of “cache” that knowledge bit (aka write me a function, class, library, whatever). In the future, the nondeterministic path now has a deterministic piece to lean on as it explores the problem space. Rinse, repeat, eventually you have a mostly deterministic system now. Leave flexibility in space where you need that nondeterminism.

Rather than telling the LLM "loop through these files", tell it "write a script to loop through these files", then hard-code that script somewhere.

The models will eventually be able to know that they need to do that to get the thing done from natural language

"The models will eventually..." Yeah but they haven't, and it's been years now. Also who cares? We have problems right now that need to be solved.

First we gave LLMs access to bash commands. Now we give them access to customized commands which they can reuse. It's English language extending its claws into deterministic programming language. Now can we please have backtracking and dynamic programming like thinking loop built into English language or such orchestration prompts.

Reply on news.ycombinator.com