ramoz

This has aged well. Paradoxically, the more capable AI gets, the more important specification becomes (or costly lack of it becomes), and the more time you spend planning and iterating on intent before you let an agent act.

Tokens and code might be cheap, but we are moving closer and closer to where we let AI operate overnight or where we want agents to operate within our actual environments in real time. The tokens or actions in these instances, have higher costs.

Right now I enjoy the labs' cli harnesses, Claude Code, and Codex (especially for review). I do a bunch of niche stuff with Pi and OpenCode. My productivity is up. Some nuances with working with others using the same AI tools- we all end up trying to boil the ocean at first- creating a ton of verbose docs and massive PRs, but I/they end up regressing on throwing up every sort of LLM output we get. Instead, we continously refine the outputs in a consumable+trusted way.

My workday is fairly simple. I spend all day planning and reviewing.

1. For most features, unless it's small things, I will enter plan mode.

2. We will iterate on planning. I built a tool for this, and it seems that this is a fairly desired workflow, given the popularity through organic growth. https://github.com/backnotprop/plannotator

  - This is a very simple tool that captures the plan through a hook (ExitPlanMode) and creates a UI for me to actually read the plan and annotate, with qol things like viewing plan diffs so I can see what the agent changed.

3. After plan's approved, we hit eventual review of implementation. I'll use AI reviewers, but I will also manually review using the same tool so that I can create annotations and iterate through a feedback loop with the agents.

4. Do a lot of this / multitasking with worktrees now.

Worktrees weren't something I truly understood the value of for a while, until a couple weeks ago, embarrassingly enough: https://backnotprop.com/blog/simplifying-git-worktrees/

I follow a very similar workflow, with manual human review of plans and continuous feedback loops with the plan iterations

See me in action here. It's a quick demo: https://youtu.be/a_AT7cEN_9I

There was a complete verification. This entire thread provides context around what I originally published - which I wouldn't recommend recreating

Thanks for the transparency. Sorry for the noise.

I think I'd be okay with a smaller, more narrative-detailed plan - not so much about verbosity, more about me understanding what is about to happen & why. There hadn't been much discourse once planning mode entered (ie QA). It would jump into its own planning and idle until I saw only a set of projected code changes.

This stemmed from me asking Claude itself why it was writing such _weird_ plans with no detail (just a bunch of projected code changes).

Claude stated: in its system prompt, it had strict instructions to provide no context or details. Keep plans under forty lines of code. Be terse.

I understand. Thank you for sharing. I didn't uncover all of this until Claude told me its specific system instructions when I asked it to conduct introspection. I'll revise the blog so that I don't encourage anybody else to do deeper introspection with the tool.

I apologize for doing this - and I agree. I will revise

I understand. Just with AI, I don't think the behavior should change so drastically. Which I understand is paradoxical because we enjoy it when it can 10x or 1000x our workflow. I think responsible AI includes more transparency and capability control.

I'll try posting again today, just because this is an active thing that I'm trying to get fixed.

There's a known bug in the VS Code native extension - hooks dont work. Somewhere in their sea of issues on GitHub, it's in there.

Well, your best bet is some type of hook that can just reject ExitPlanMode and remind Claude that he's to stay in plan.

You can use `PreToolUse` for ExitPlanMode or `PermissionRequest` for ExitPlanMode.

Just vibe code a little toggle that says "Stay in plan mode" for whatever desktop you're using. And the hook will always seek to understand if you're there or not.

  - You can even use additional hooks to continuously remind Claude that it's in long-term planning mode. 

*Shameless plug. This is actually a good idea, and I'm already fairly hooked into the planning life cycle. I think I'll enable this type of switch in my tool. https://github.com/backnotprop/plannotator

The OpenCode plan experience has been pretty bad (the community has accepted this, at least on Discord). The community's adopted a handful of plugins to make the experience better, and also guardrail when the agent switches versus doesn't

Great job with the tool.

If the data includes every contractor that competed for an award & lost, that is data is typically not public.

The first one is literally a well-known massive corporation

It is all public.

It's missing data for sure - at least on the awarded side. (Which is easy to vet because it's all public)

https://www.usaspending.gov/search?hash=181b0ab9a8cc9f30fbed...

The deterministic context system is intuitive and well-designed. That said, there's more to consider, particularly around user intent and broader information flow.

I created the hooks feature request while building something similar[1] (deterministic rails + LLM-as-a-judge, using runtime "signals," essentially your context). Through implementation, I found the management overhead of policy DSLs (in my case, OPA) was hard to justify over straightforward scripting- and for any enterprise use, a gateway scales better. Unfortunately, there's no true protection against malicious activity; `Bash()` is inherently non-deterministic.

For comprehensive protection, a sandbox is what you actually need locally if willing to put in any level of effort. Otherwise, developers just move on without guardrails (which is what I do today).

[1] https://github.com/eqtylab/cupcake

Level4 is most interesting to me right now. And I would say we as an industry are still figuring out the right ergonomics and UX around these four things.

I spend a great deal of my time planning and assessing/reviewing through various mechanisms. I think I do codify in ways when I create a skill for any repeated assessment or planning task.

> To be clear, planning as a general practice isn't going away. It's just changing shape. For newer practitioners, plan mode remains the right entry point (as described in Levels 1 and 2). But for complex features at Level 7, "planning" looks less like writing a step-by-step outline and more like exploration: probing the codebase, prototyping options in worktrees, mapping the solution space. And increasingly, background agents are doing that exploration for you.

I mean, it's worth noting that a lot of plan modes are shaped to do the Socratic discovery before creating plans. For any user level. Advanced users probably put a great deal of effort (or thought) into guiding that process themselves.

> ralph loops (later on)

Ralph loops have been nothing but a dramatic mess for me, honestly. They disrupt the assessment process where humans are needed. Otherwise, don't expect them to go craft out extensive PRD without massive issues that is hard to review.

  - It would seem that this is a Harness problem in terms of how they keep an agent working and focused on specific tasks (in relation to model capability), but not something maybe a user should initiate on their own.

I don't think anybody at Meta involved in the aquisition must be an avid OpenClaw user or developer.

Moltbook was more of a meme - agents mostly orchestrated by users in the background.

Not something with motion like OpenClaw itself (with a real community).

Skills is the right abstraction.

Side project - plan mode and code review annotations for coding agents (ui that integrates via hooks): https://github.com/backnotprop/plannotator

Main gig: Trusted agents. We just shipped hardware based signing to web bot auth protocol.

Where are we at with SOTA or reliable prompt injection detection mechanisms?

My nervous system is damaged, unfortunately, therefore it's still here! I appreciate the tips though!

For me, it's a very distinct ringing. Like in the movies when there's some explosion/shock scene and there's a very persistent ding that happens after the explosion or whatever - it's what I hear constantly.

> Claude Code style plan mode to be a bit restrictive

Hey thats why i built plannotator: https://github.com/backnotprop/plannotator

I like staying within Claude Code for orchestrating its plan mode, but I needed a better way to actually review the plan, address certain parts, see plan diffs, etc all in a better visual way. The hooks system through permissionrequest:exitplanmode keep this fairly ergonomic.

see it in action: https://www.youtube.com/watch?v=a_AT7cEN_9I

Don’t let it get to you like this.

Same experience here.