Hacker News Reader

Comment by vrosas | Hacker News Reader

The problem is, when the context window grows, Claude tends to forget these kinds of rules. It will then do whatever it wants. I had to outright ban comments in the global claude.md, the local claude.md AND write a hook to catch any that still slipped through.

View on HN

I think people really need to focus more on working with limited contexts rather than trying to work around it. I really try to keep my sessions as short as possible and it helps a ton with keeping Claude (et al) focused.

Specifically, I like the "canary" trick that people have discussed where you add a small, innocuous rule to your CLAUDE.md like "When responding to me, start every sentence with my name." so that when Claude stops doing this, you know you've used way too much context and need to start a new session.

This or you just repeat the initial prompt every 200k tokens

Which gets you to the point where the whole thing is.. still unreliable. Generative text engines are going to generate. This calls for real enforcement in deterministic pre-edit hooks.

And here is where naive people will say something like "Why do I care if robots shit all over the codebase? Code is for machines, I don't expect to deal with it much now". But really externalized CoT like this confuses machines too, wastes tokens, and eventually wastes exponentially many tokens. Agents tend to think it's more real grounding than prompts are, even for comments-in-code. One bad comment poisons everything, then gets copied around as a ground-truth assumption everywhere. Hooks are more real to them than prompts or comments, and even then if you add enforced limits and tell them to externalize CoT ONLY in scratch task-tracking docs.. they will violate comment-enforcement hooks about 25% of the time. That tells you everything you need to know: even with constant reinforcement, they just really want to break this kind of rule.

My take on this is they are a tool to help speed up your work they are not meant to produce finished work. Humans produce finished work. LLM will never be deterministic cause their entire value is that they are generalizable.

I hear that, sort of, but here's the thing. Using AI at scale means AI needs to be nearly perfect about not shitting where they eat. That's the subject matter of the whole thread

So the options are a) being a really aggressive stickler for generative hygiene with deterministic rules, b) being massively wasteful about hiring a few machine janitors for every machine coder, or c) humans become the machine's janitor. If I haven't missed an option.. only the first option seems reasonable here.

"AI at scale" is just a euphemism for slop. Current gen AI can augment human engineers but not outright replace them.

This isn't really responsive to what I'm saying or what the discussion here is about, but if you insist. Would you describe lots of AI augmenting lots of human engineers as perhaps.. AI at scale?

Yup. If only there were a task completion hook that could be set to fire prior to rendering terminal output. That would more handily address all these issues, as we could simply enforce output style rules that way.

The current output style does work, but it’s a Sisyphean task to tweak it constantly only to find out that CC adhere’s to only 75% of it, no matter what…

I set my env up so I can see the exact context used in CC CLI, and then once I get over about 40% ctx used I have it handoff to a new, fresh session. Nothing good comes from running above, say 60% of your context window. Coincidently, I usually have good results with CC. I never compact a session ever.

This is the way. Mine looks like `ctx: 0k / Fable 5 high`. When ctx ⪆ 200k, time to start a new session.

```statusline-command.sh

    #!/bin/sh
    input=$(cat)

    model=$(echo "$input" | jq -r '.model.display_name // empty' | sed 's/ context)/)/')
    effort=$(echo "$input" | jq -r '.effort.level // empty')
    used=$(echo "$input" | jq -r '.context_window.total_input_tokens // empty')
    out=""

    [ -n "$used" ] && out="ctx: $(( used / 1000 ))k"
    [ -n "$model" ] && out="$out / $model"
    [ -n "$effort" ] && out="$out $effort"

    # bright yellow + bold
    printf '\033[1;93m%s\033[0m' "$out"

```

Yea, I've started making it write linters to check the code that goes out. Anything that can be deterministically measured, gets added to it once we lock it down.

> The problem is, when the context window grows,

You know the problem; then why not address it? Does Compacting the context not help?

Compacting context compacts context. So Claude forgets a lot during compaction.

Compacting mostly gets rid of reasoning tokens, and honestly it would be nice of reasoning tokens did not constantly follow every follow up query. Asking even a simple/trivial question can have Claude use thousands of tokens. Compacting is good for getting rid of those.

I've had Claude immediately fall back to its usual verbose style immediately after compaction.

To be fair, I've had it do that immediately after re-reading the output style instructions, too.

My chat history is filled with "Yes, I broke the language rule. Let me rephrase that and update my memory. — You already have that in memory — Yes, true, I ignored that" (because "Memory" is a yet another .md file)

Claude Code supposedly supports a "post-compaction" hook, so you could have it automatically run the prompt "We just compacted the context, quickly refresh yourself on the rules in CLAUDE.md etc..".

Depending on what you've got in those files, maybe that will just use up all the context again though.

> supposedly supports a "post-compaction" hook, so you could have it automatically run the prompt

Keyword "supposedly" :)

I've had it in my settings forever, and still...

Asking it to analyse and fix the issue it produced a plausible "my training supercedes/overrides settings especially if triggered by certain words in the phrase" (paraphrasing the long text)

> Keyword "supposedly" :)

> I've had it in my settings forever, and still...

Checks out! I've never used it my self, so it I figured it likely didn't work at all.

Compacting the conversation almost never helps. It is uniformly worse than starting over with fresh context, or rewinding to a last-known-good state. It only exists because it increases engagement.

This does not match my experience. I use long-running orchestrator sessions. Each orchestrator is in charge of planning, writing kick-off prompts for implementers, answering questions from those implementers, doing code reviews and providing feedback, and answering side questions from me when I have them.

Depending on the initiative I might compact a session a dozen times, sometimes more. It is lossy, and the session certainly tends to forget earlier bits as more compactions happen, but overall it's a much better experience than starting fresh and having to re-explain everything.

The only time I compact is if the session goes wildly off-course and the context gets polluted with off-topic conversations.

Also worth noting: with Claude Code you can provide custom instructions when compacting, and instruct the LLM that is in charge of compacting the session to prioritize the retention of specific bits. It can help a lot.

Compaction is a main cause of this problem.

Hooks is the way.

Intermittent nudges

I have a post-edit hook that asks the model if this edit introduced any comments, and if it did, remove them. This was the only reliable way to get it to stop that I found.

Reply on news.ycombinator.com