Ask HN: How do you maintain depth of understanding and velocity when using AI?

Apollo 13 disaster highlighted the need for digital twins. NASA gave engineers only a table of junk available to astronauts inside the spacecraft to make a CO2 scrubber (so they would not die). This resulted in a comprehensive set of engineering and troubleshooting best practices.

Geekatron made a Claude plugin out of that framework. It runs Strawman, Steelman, Red Team, Blue Team, Engineering, etc specialists against your design/code. Provides analysis, criticism, recommendations, and 'best practices' you have probably seen individually into a coherent/rigorous process.

For deeper dive & an example: DankOverride forked an OpenTelemetry version of the Geekatron/Jerry plugin. It runs within an open-source "Shoebox" that simulates networks for troubleshooting. This shoebox can also be used to help critique how well a monitoring software platform passes the "SpatialObservability" Three Tests (Continuity, Proximity, Memory).

One of the things I usually do before reviewing every line of code myself is have a separate agent review it. I also spend a lot more tokens on planning than I do on implementation. I will have my agents help me groom tickets in a way that will explode them into a whole series of much smaller tickets that the agents themselves can handle as far as planning and implementation in small little bits and and then review is based on much more distinct pieces that are easier to digest as a reviewer.

I find a lot of things that make me better and make me more effective at agentic development are also things that would make me better or more effective at leading a team of very junior engineers. Spend more time making the instructions clear. Spend more time breaking tasks into smaller tasks and linking them together properly.

If you spend a little bit more time on project management, you can spend much less time arguing with your agents.

To ensure max utilization of my attention, I do the following steps when working on new features:

  - Straight brain dump for ideas. I don't spend time organizing thoughts. Sometimes I don't even write prompts with right English syntax. But AI will understand most of them.
  - Ask AI to grill me on the missing details in the design. Usually it's the most painful step.
  - AI writes design doc. I skim through it. Give some feedback.
  - Let AI implement the feature. Recent AI models can usually complete the full feature without my intervention, as long as the design doc is solid.
  - If I feel there are unclear part in the implementation, I ask AI to write explainer doc.

The things many people probably don't do is:

  - The reverse grilling. Our description on the thing we want is often incomplete. If we don't make AI to ask enough clarification questions, there will be misalignment.
  - Now I ask AI to write all docs in html. It takes more tokens and time to finish, but much easier to read and understand, because of the richer layout and sometimes the interactivity of html/js.

I used claude artifacts to give feedback about the html design doc directly to the agent, and later on, developed my own tool (https://github.com/hyperlogue/r3) to do the same thing but for all kinds of agents.

yea its tough, when it comes to frontend you can go really fast because reviewing is easy since its something you can visually pick. for backend its exactly as you described, perhaps the ideal thing you could do is speed up your review by using ai rather than going over every line or spend more time planning and writing the spec.

I was the same when i just started using codex, i dont know the exact time but at some point i just stopped reviewing, dumb but the more i used it the more lazy i became.

Right, I do use AI for review and find it helpful most of the time (assuming it doesn't decide to be unnecessarily pedantic). I have found it struggles to reason through architectural implications or subtle performance problems when there isn't a specific linter for it, though.

For example, I'm instantiating something N times with this change. Each instantiation is fast, but this is a hot path. It's not clear whether that's safe or if it needs to be gated which is something a reviewer would ideally flag.

I suppose the counter-argument most people would make now is that if the AI didn't call it out, it's probably not likely enough of an issue to focus on -- even if it does wind up becoming a problem later.

something that works well to me is trying to make very compact PRs, with just one intention, and not divagate into multiple ideas at the same time.

also at my company we use Revix AI for reviews which do not remove the human but gets most of the little things and allows us to keep proper code in a team where most of juniors and mids use AI without much control

FWIW, I’m in the same minority(?) as you. I suspect that others just have lower standards of rigor.

Do you find yourself still trying to use AI tools for the synthesis portion of writing/coding, or have you gone back to the 'manual' ways?

I've pretty much completely dropped it for writing (but still use it for catching issues with clarity or logical flow afterwards). I've yet to get away from it for coding. Perhaps I keep trying with coding because I feel I'm doing something wrong (and because for a a little my performance was tied to usage of it...).

I have no pressure to use LLMs for agentic coding, so I haven’t tried that hard, to be honest. I use LLMs to generate initial code drafts and to perform logical refactors (those which classic mechanical refactoring tools are unsuitable for) that I then touch up or revise manually. Basically, areas where it actually saves time while still fully controlling the design of the code and reasoning through all aspects of the implementation. I don’t see how agentic coding can save time without giving up some level of diligence, coherence, and attention to detail, which I’m not willing to do.

I am so tired of prompting in chats. But we've got to provide input and intent somehow. Here's the strategy I'm trying...

Instead of prompting, I hack on code in my editor.

My harness gather everything it needs from the local context - my git diff, my open editor buffers, etc. to assess what I've been doing. No chat. This is fed into phase 2 which tries to guess my intent. Then phase 3, it presents a plan to complete the work. The only user interaction is reviewing the plan and typing yes or no.

The quality of the plan of course depends on the quality of my uncommitted ideas. As it should be. If the plan goes off the rails, it's my fault. Do not chat your way to a solution! Abort the session and continue fleshing out the idea in source code/markdown.

The reason I like this is it forces me to at least take a stab at the work. I treat the AI like a relief pitcher to come in and close out the game.

I can't say this is the way to hyper productivity. I'm still slow. But at least I'm spending exactly 0 hours a day arguing with an LLM!

I like this, it feels like the right balance. I still struggle to define when the handoff point is, but I think maybe it's at the same point as when I would hand it off to a junior engineer: when I've sketched almost everything I need to be quite confident in my approach (and if I'm wrong, I'll find out and try again).

i think you need to constrains AI with the help of powerful prompt