The whole thing was borne out of wanting to keep costs low! My favoured approach (last time I was doing this) is using only a tiny sliding context window based on message pairs, rather than tokens, and only for the agents that need it. Amending prose style, for example, shouldn't need context beyond the message it's working on, and then its system prompt.

For the models that require context, I personally found combining a tiny sliding window with a lazy version of the "Recursive Language Models" approach broke immersion least often and had a significantly lower cost. That + the "Id noise" + the strict agents also allowed cheaper models to overperform for me personally.

My lazy version of the RLM approach is basically just giving the agent a grep tool across the full message history & "lore" documentation created by agents, combined with repeated, low-context turns, and a "submit answer" tool for when it felt like it had finished working.

When I looked at the internals of what each agent turn looked like, it did look like a complete mess - but the context window only needs to surface the things it actually needs to know each turn.

Short outputs help a lot with immersion, too - brevity means there is a lot less you can get wrong, and also aids response time & cost.

It does take me an awful lot of prompt tuning to get what I want creatively from LLMs in any format, especially weaker models working in this chain, but I think that's likely always going to be true. Art can have rules, but that doesn't make it science :-)

The RLM approach is detailed here, and I've found it really useful for any cost-sensitive/long-context task: https://alexzhang13.github.io/blog/2025/rlm/