Plain Claude, ask it to write a plan, review plan, then tell it to execute still works the best in my experience.
The reality is that if you actually know what you want, and can communicate it well (where the productivity app can be helpful), then you can do a lot with AI.
My experience is that most people don't actually know what they want. Or they don't understand what goes into what they want. Asking for a plan is a shortcut to gaining that understanding.
So, to really create something new that I care about, LLMs don't help much.
They are still useful for plenty of other tasks.
This particular skill is not great.
And i drawn a tarot card and the card refuted your gut feeling.
Joking asside, there is no reason to suspect that an LLM is telling you correct information about how best to use an LLM. The way to confirm your thesis is by running experiments, or finding someone who has already done them. It is a valid move to use an LLM to find primary sources, but in itself the LLM is not an authority you could or should trust.
I can’t tell you how many times I have a CS student in my office for advising and they tell me they only want to take technical courses, because anything reading or writing or psychology or history based is “soft”, unrelated to their major, and a waste of their time.
I’ve spent years telling them critical reading and expressive writing skills are very important to being a functioning adult, but they insist what they need to know can only be found in the Engineering college.
Engineers who lack soft skills cannot be effective in team environments.
To be fair, LLMs can be quite useful for quickly finding the correct place in TFM to look when you don't necessarily have a function or feature name to go on.
1000%. This is why people whose job it was to figure out how to make a thing are thriving with AI tools, and those who operate in the conceptual/abstract are flailing and frustrated with it. (it really is a mirror in this case, and the frustration they have is unknowingly directed at themselves)
We used to have the very difficult task of producing working scalable maintainable code describing complex systems which do what we need them to do.
Now on top of it we have the difficult task of producing this code using constantly mutating complex nondeterministic systems.
We are the circus bear riding a bicycle on a high wire now being asked to also spin plates and juggle chainsaws.
Maybe singularity means that time sunk into managing LLMs is equal to time needed to manually code similar output in assembly or punch cards.
skills that teach the agent how to pipe data, build requests, trace them through a system and datasources, then update code based on those results are a step function improvement in development.
ai has fundamentally changed how productive i am working on a 10m line codebase, and i'd guess less than 5% of that is due to code gen thats intended to go to prod. Nearly all of it is the ability to rapidly build tools and toolchains to test and verify what i'm doing.
What sort of skills are you referring to?
Skills are crazy useful to tell Claude how to debug your particular project, especially when you have a library of useful scripts for doing so.
Claude is kind of decent at doing "when in Rome" sort of stuff with your codebase, but it's nice to reinforce, and remind it how to deploy, what testing should be done before a PR, etc.
I'd love to know how this skill was phased.
It's certainly an iterative process but it gets better every iteration.
And also we know why: effective context depends on inout and task complexity. Our best guess right now is that we are often between 100k to 200k effective context length for frontier, 1m NIHS type models
So I naturally felt the need to (tell Claude to) build a MCP for this accounting API, and now I ask it to do accounting tasks, and then it just does them. It's really ducking sweet.
Another thing I did was, after a particularly grueling accounting month close out, I've told Claude to extract the general tasks that we accomplished, and build a skill that does it at the end of the month, and now it's like having a junior accountant in at my disposal - it just DOES the things a professional would charge me thousands for.
So both custom project MCPs and skills are super useful in my experience.
(I'm genuinely asking)
To give you a small taste: you need to issue an electronic invoice for each unique customer, and submit on the fly the tax authority - but these need to correlated monthly with the money in your business bank account. The paid invoices don't just go into your bank account, they are disbursed from time to time by the payment processor, on random dates that don't sync with the accounting month, so at end of month you have to have correlate precisely what invoice is paid or not. But wait, the card processor won't just send you the money in a lump sum, it will deduct from each payment some random fee that is determined by their internal formula, then, at the end of each month, add all those deducted fees (even for payments that have not been paid to you) and issue another invoice to you, which you need to account for in you books as being partially paid each month (from the fees deducted from payments already disbursed). You also have other payment channels, each with their fees etc. So I need to balance this whole overlapping intervals mess with all sort of edge cases, chargebacks and manual interventions I refuse to think about again.
This is one example, but there are also issues with wages and their taxation, random tax law changes in the middle of the month etc. The accountant can of course solve all this for you, but once you go a few hundred invoices per month (if you sell relatively cheap services) you are considered a "medium" business, so instead of paying for basic accounting services less than 100€ per month (have the certified accountant look over your books and sign them, as required by law), you will need more expensive packages which definitely add up to thousands in a few months.
Go be an entrepreneur, they said.
Claude and an mcp and skill is plain to me. Writing your own agent connecting to LLMs to try to be better than Claude code, using Ralph loops and so on is the rabbit hole.
Though, you get such a huge bang from customizing your config that I can easily see how you could go down that slippery slope.
I’m still new to this, but the first obvious inefficiency I see is that I’m repeating context between sessions, copying .md files around, and generally not gaining any efficiency between each interaction. My only priority right now is to eliminate this repetition so I can free up buffer space for the next repetition to be eliminated. And I don’t want to put any effort into this.
How are you guys organizing this sort of compounding context bank? I’m talking about basic information like “this is my job, these are the products I own, here’s the most recent docs about them, here’s how you use them, etc.” I would love to point it to a few public docs sites and be done, but that’s not the reality of PM work on relatively new/instable products. I’ve got all sorts of docs, some duplicated, some outdated, some seemingly important but actually totally wrong… I can’t just point the agent at my whole Drive and ask it to understand me.
Should I tell my agent to create or update a Skill file every time I find myself repeating the same context more than twice? Should I put the effort into gathering all the best quality docs into a single Drive folder and point it there? Should I make some hooks to update these files when new context appears?
- A well-structured folder of markdown files that I constantly garden. Every sub-folder has a README. Every files has metadata in front-matter. I point new sessions at the entry point to this documentation. Constantly run agents that clean up dead references, update out of date information, etc. Build scripts that deterministically find broken links. It's an ongoing battle.
- A "continuation prompt" skill, that prompts the agent to collect all relevant context for another agent to continue
- Judicious usage of "memory"
- Structured systems made out of skills like GSD (Get Shit Done)
- Systems of "quality gate" hooks and test harnesses
For all of these, I have the agent set them up and manage them, but I've yet to find a context-management system that just works. I don't think we understand the "physics" of context management yet.
Great docs help you, your agents, your team and your customers.
If you’re confused and the agent can’t figure it out reliably how can anyone?
Easier said than done of course. And harder now than ever if the products are rapidly changing from agentic coding too.
One of my only universal AGENTS.md rules is:
> Write the pull request title and description as customer facing release notes.
One quick win I’ve thought could bridge this is updating our docs site to respond to `Accept: text/markdown` requests with the markdown version of the docs.
1. I have many and sometimes contradictory workflows: exploration, prototyping, bug fixing debugging, feature work, pr management, etc. When I'm prototyping, I want reward hacking, I don't care about tests or lint's, and it's the exact opposite when I manage prs.
2. I see hard to explain and quantify problems with over configuration. The quality goes down, it loses track faster, it gets caught in loops. This is totally anecdotal, but I've seen it across a number of projects. My hypothesis is that is related to attention, specifically since these get added to the system prompt, they pull the distribution by constantly being attended to.
3. The models keep getting better. Similar to 2, sometime model gains are canceled out by previously necessary instructions. I hear the anthropic folks clear their claude.md every 30 days or so to alleviate this.
Context still matters and I'll never stop implementing things in small slices instead of trying to one-shot.
Working on an unspecified codebase of unknown size using unconfigured tooling with unstated goals found that less configuration worked better than more.
* Claude trying to install packages into my Python system interpreter - (always use uv and venvs)
* Claude pushing to main - (don't push to main ever)
* When creating a PR, completely ignoring how to contribute (always read CONTRIBUTING.md when creating a PR)
* Yellow ANSI text in console output - (Color choices must be visible on both dark and light backgrounds)
Because I got sick of repeating myself about the basics.
For all of my projects task build runs linters, tests and builds the project with as little output as possible on a happy path.
This catches a bunch of "it's a pre-existing issue" stuff from Claude. Sometimes I ask it to run build first, then start implementing just so that it can prove to itself that no, it wasn't a pre-existing issue, you broke something.
my only machine-specific config is overriding haiku usage with sonnet in claude code. i outline what i want in linear, have claude synthesize into a plan and we iterate until we're both happy, then i let it rip. works great.
then one of my juniors goes and loads up things like "superpowers" and all sorts of stuff that's started littering his PRs. i'm just not convinced this ricing of agents materially improves anything.
The proper way to do this is find a personal pain point, figure out how to fix it, fix it, and then continue.
That's how I built my own system, zero skills, just a git submodule with shared guides how to do stuff the way _I_ like it. I can just refer any agent to read that directory and they'll usually get it on the first go.
All the fancy frameworks are vibe coded, so why could they do better than something you do by yourself?
At most get playwright MCP in so the agent can see the rendered output
But beyond that, I just ask it what I want it to ask, and that's it. I'm not convinced that putting more time into building the "toolbox" will actually give me significant returns on that time.
I do think that some of this (commands, skills, breaking up CLAUDE.md into separate rules files) can be useful, but it's highly context-dependent, and I think YAGNI applies here: don't front-load this work. Only set those up if you run into specific problems or situations where you think doing this work will make Claude work better.
e.g. spend time creating a skill about how to query production logs
But for some projects there will be things Claude doesn’t know about, or things that you repeatedly want done a specific way and don’t want to type it in every prompt.