dhorthy

I don’t think anyone serious would recommend it for serious production systems. I respect the Ralph technique as a fascinating learning exercise in understanding llm context windows and how to squeeze more performance (read: quality) from today’s models

Even if in the absolute the ceiling remains low, it’s interesting the degree to which good context engineering raises it

there are hundreds of useful resources, including many linked in the article itself

the note about the crypto token was intended to “okay this is now hype slop and it’s time to move on”

I read it. i agree this is out of touch. Not because the things its saying are wrong, but because the things its saying have been true for almost a year now. They are not "getting worse" they "have been bad". I am staggered to find this article qualifies as "news".

If you're going to write about something that's been true and discussed widely online for a year+, at least have the awareness/integrity to not brand it as "this new thing is happening".

engineers always want to re write from scratch and it never works.

a tale as old as time - my second job out of college back in like 2016, I landed at the tail end of a 3-month feature-freeze refactor project. was pitched to the CEO as 1-month, sprawled out to 3 months, still wasn't finished. Non-technical teams were pissed, technical teams were exhausted, all hope was lost. Ended up cutting a bunch of scope and slopping out a bunch of bugs anyway.

i had the privilege of working w/ some incredible eng leaders at my previous gig - they were very good at working both upwards and downwards to execute against the "50/50" rule - half of any given sprint's work is focused on new features, and half is focused on bug fixes, chores, things that improve team velocity.

the people yearn for refactoring

I think the key here is “if X then Y syntax” - this seems to be quite effective at piercing through the “probably ignore this” system message by highlighting WHEN a given instruction is “highly relevant”

agree - i've had claude one-shot this for me at least 10 times at this point cause i'm too lazy to lug whatever code around. literally made a new one this morning

For the record I do think the AI community tries to unnecessarily reinvent the wheel on crap all the time.

sure, readme.md is a great place to put content. But there's things I'd put in a readme that I'd never put in a claude.md if we want to squeeze the most out of these models.

Further, claude/agents.md have special quality-of-life mechanics with the coding agent harnesses like e.g. `injecting this file into the context window whenever an agent touches this directory, no matter whether the model wants to read it or not`

> What people often forget about LLMs is that they are largely trained on public information which means that nothing new needs to be invented.

I don't think this is relevant at all - when you're working with coding agents, the more you can finesse and manage every token that goes into your model and how its presented, the better results you can get. And the public data that goes into the models is near useless if you're working in a complex codebase, compared to the results you can get if you invest time into how context is collected and presented to your agent.

> If I tell them exactly how to build something the work needed to review the resulting changes is a whole lot less taxing.

Totally matches my experience- the act of planning the work, defining what you want and what you don’t, ordering the steps and declaring the verification workflows—-whether I write it or another engineer writes it, it makes the review step so much easier from a cognitive load perspective.

would "wrote" be more appropriate than "shipped"?

there is a portion in the article where I talk about how our hadoop refactor completely failed

I have had a user story and a research plan and only realized deep in the implementation that a fundamental detail about how the code works was missing (specifically, that types and sdks are generated from OpenAPI spec) - this missing meant the plan was wrong (didn’t read carefully enough) and the implementation was a mess

My advice - never use compact, always stash your context to Md or a wordy git commit message and then clear context

You want control over and visibility into what’s being compacted, and /compact doesn’t do great on either

You don’t think early c programmers spent a lot of time reading the assembly that was produced?

I appreciate the share. Yes as I said it was a pretty dang uncomfortable to transition to this new way of working but now that it’s settled we’re never going back

File size matters if you don’t have strategically placed “read the entire file” instructions for certain parts of the workflow (we do)

yeah flat, simple code is good to start, but I find I'm still developing instincts around right balance between "when to let duplicate code sprawl" vs. "when to be the DRY police".

i 100% agree - the folks who are best at ai-first engineering, they spend 3 days designing the test harness and then kick off an agent unsupervised for 2+ days and come back to working software.

not exactly valuable as guidance since programming languages are very easy to verify, but the https://ghuntley.com/ralph post is an example of whats possible on the very extreme end of the spectrum

yeah its kinda funny how some bigger more sophisticated eng orgs that would be called "slow and ineffective" by smaller teams are actually pretty dang well set-up to leverage AI.

All because they have been forced to master technical communication at scale.

but the reason I wrote this (and maybe a side effect of the SF bubble) is MOST of the people I have talked to, from 3-person startups to 1000+ employee public companies, are in a state where this feels novel and valuable, not a foregone conclusion or something happening automatically

letting people pick their own editors is a zirp phenomenon

yeah if you read our create_plan prompt, it sets up a 3+ phase back and forth soliciting clarifying questions before the plan is built!

Claude code is Claude code, whether you use in cursor or not

Codex and Claude code are neck and neck, but we made the decision to go all in on opus 4, as there are compounding returns in optimizing prompts and building intuition for a specific model

That said I have tested these prompts on codex, amp, opencode, even grok 4 fast via codebuff, and they still work decently well

But they are heavily optimized from our work with opus in particular

i think the general take away for all of this is the model can write the code but you still have to design it. I don't disagree with anything you've said, and I'd say my advice is engage more, iterate more, and work in small steps to get the right patterns and rules laid out. It wont work well on day one if you don't set up the right guidelines and guardrails. That's why it's still software engineering, despite being a different interaction medium.

thank you for the feedback! themes are hard. Update going out now

we can come up with something better :)

i think that was one of the key reasons we built research_codebase.md first - the number one concern is

"what happens if we end up owning this codebase but don't know how it works / don't know how to steer a model on how to make progress"

There are two common problems w/ primarily-AI-written code

1. Unfamiliar codebase -> research lets you get up to speed quickly on flows and functionality

2. Giant PR Reviews Suck -> plans give you ordered context on what's changing and why

Mitchell has praised ampcode for the thread sharing, another good solution to #2 - https://x.com/mitchellh/status/1963277478795026484

if you read further down, I acknowledge this

> While the cancelation PR required a little more love to take things over the line, we got incredible progress in just a day.