Hacker News Reader

Comment by wren6991 | Hacker News Reader

Given how poorly LLMs do with writing prompts for LLMs, I'm not sure I'd trust their judgement in designing a language for LLMs.

> and the runtime requires explicit permission to touch the filesystem, network, etc

This feels like more of an OS problem (or library problem) than a language problem.

> Run one program against many worlds. The same code can run against the real network, a scripted fake, a recording of last week's traffic, or a probability model of how servers usually behave

How is the "world" model different from plain dependency injection?

View on HN

> How is the "world" model different from plain dependency injection?

In addition to what the other comment said, this "world" model is great for hermetic testing of complex code, LLM written or not. We've seen existing projects that intercept the OS level syscall for testing, replayability, etc. Building it into the language runtime, hopefully with better ergonomics from the start than a syscall, would be a welcome addition broadly.

Dependency injection provides the same hermetic-testing capability, so that's not really an answer to OP's question. Effect-as-world-model is, in this specific way, just a special argument slot that only accepts dependency-injected functions.

What effects provide beyond DI is entirely in their ability to abort (resume zero times) or (in the case of multi-shot effects) resume multiple times. An effect that resumes exactly once is structurally the same as a dependency injected lambda.

> What effects provide beyond DI is entirely in their ability to abort (resume zero times) or (in the case of multi-shot effects) resume multiple times

Thanks, this answers my question and does sound handy. Also I do see the value in having more problem-shaped versions of a general construct.

>This feels like more of an OS problem (or library problem) than a language problem.

And from my perspective it's the exact opposite. You can apply a sandbox on the OS level and the library you call will crash at runtime. In your model you now get to harass the library author "hey you didn't tell me you do I/O" after it is already too late.

Crashing instead of raising an exception or bubbling a return code sounds like a legitimate bug ticket IMO.

One reason I'm in favour of handling capabilities in the OS is so we can stop having trivial symlink traversal and /.. traversal bugs in path filters (or indeed more complex bugs in the face of FS-specific linking primitives).

> Given how poorly LLMs do with writing prompts for LLMs, I'm not sure I'd trust their judgement in designing a language for LLMs.

Yeah agree, what I really want to see now, is a programming language for LLMs, designed by a human (although code could still be LLM-made I suppose), and see how both of them fare in various scenarios.

Why do you think LLMs write prompts for LLMs badly? I use LLMs to write and refine prompts all the time. The prompts seem to come out very good. What are you basing this on?

Generally excessively verbose and failing to account for LLM failure modes. Example: when dispatching a subagent with an instruction prompt, I see the agent refer to "phase 5 of the plan" without actually pointing to the plan file it's referencing. The subagent says something to the tune of "phase 5, sure thing boss" and goes and generates garbage because it never reads the plan to find out what "phase 5" entails.

If I ask an agent to maintain agent-facing docs like AGENTS.md then similarly it ends up with low information density and not really saying what I want it to say. Something I thought was unequivocal ends up leaving the agent with far too much scope for interpretation and judgement calls.

Perhaps I'm just Prompting It Wrong (TM) but I do find LLMs lack the theory of mind to realise that other LLM instances don't know the things that the current instance knows, and make poor calls on what to include and what not to include. It's a bit hand-wavey, but I was wondering if the same issue might transfer to language design.

I've done a version of "world" for Sigil, a programming language I was kinda doing but stopped, also for agents. LLM generated article here https://inerte.github.io/sigil/articles/worlds-not-mocks/

But basically world is a bit more narrow, that moment where your code touches the outside world (logging, http, etc), you can swap that. It's sorta like DI but deliberately narrower, only the moments where code touches the outside world are swappable.. With DI in theory you can replace anything, which has its benefits, but at least personally I am not a big fan of mocks, except when they touch the outside world. So that's what's replaceable.

Reply on news.ycombinator.com