Hacker News Reader

Comment by mtlynch | Hacker News Reader

OP here!

I'll admit a lot of bias because I think design docs are extremely useful, but I find that when people hate design docs, it's almost always for one of two reasons:

1. The developer has worked on teams where design docs are viewed as a pointless ritual, so authors treat them as a pointless requirement and write bad docs and their teammates view them as pointless so they don't bother giving useful feedback, reinforcing everyone's belief that they're a pointless ritual.

2. The developer does not like other people questioning their engineering choices, and they know that it's harder for their teammates to push back on finished code than a design doc. Investing in the implementation before design changes the calculus to bias in favor of whatever's already implemented rather than what would have been the ideal implementation. Plus, it's harder for the team to review design decisions of 10k LOC than a 5-page design doc.

View on HN

I think your framing is fair here. But I'd like to offer an even more complicated/nuanced take:

Designing in a group can be very difficult, and doing it well is a skill set that most people don't naturally have.

I think this explains your point 1. Why do people view designed docs as pointless? Because they really don't have a vision or model for what and effective and healthy collaborative design process would look like.

> Designing in a group can be very difficult, and doing it well is a skill set that most people don't naturally have

Not only is it a skill every participant needs to have, they also all need to have a similar amount of competence and knowledge about the domain as well as the current implementation, otherwise it's mostly pointless ime.

But if all ven diagram circles overlap ... It is nice. I can count the times this materialized (in my professional life) on one hand. So I'm generally more towards the "make a prototype, then explain it to the others. Either it's the baseline for the discussion or the illuminating event that clears up wherever this approach works with that team.

I am on the design-doc team and I'd like to add the 3rd category:

3. The developer is not able to imagine how the design works because s/he has no capability for abstraction. Needs to go hands-on.

I might add that writing the doc is not the purpose, the purpose is to think it, challenge it and share it.

The design "doc" is needed, but not the static doc for printers, a more dynamic one where you don't need a 50 pages doc with lots of links between pages, but a very good dynamic diagram with some text.

YES - and 3 the the assumption that it needs to be a certain length before it can be considered a design document. Esp. in the age of AI a little guidance to brainstorm on before a random project prompt goes a long way. (Not saying people that don't use design docs just code away thoughtlessly) - experience goes a long way too that's why there's success stories with and without design docs.

1b: the docs are a pointless ritual that sometimes ends up taking multiple times longer than implementing, and if too many people see it they start asking things like "what are your KPIs" and "when are your deliverables synergized" and "have you written the oncall runbooks yet? what's the protocol spec for that?" and "what's your projected uplift". if it's less than 5 pages of text, you're dinged on perf because your documents aren't detailed enough. for projects like "we should add a small in-memory cache to this slow area".

so you're best off writing a small one that you do your best to hide, and make a few fancy ones per season with graphs and absurd will-never-be-implemented details for the higher-ups to be distracted by.

So you need to believe in design documents, and if you don't like them you're probably trying to manipulate people? But have you justified design documents?

I can think of situations where design document(s) would be a clear use, and I think that would be a better way to respond.

Num. 2 especially relatable. A good mark of high quality professional is if he presents his plan before execution to hear feedback and comments - even if they are totally against his original idea, and he can then take this feedback and incorporate effectively in a re-design.

This does not always work.

Often, the group members presented to, do not have the required knowledge to critique the design document.

I have seen people proposing (demanding even) changes based on vague feelings and their need to be seen as contributing.

Once they say something, it is out there, now it is the poor presenter who has to refute it or accept modifications to the design.

If they don't have the required knowledge before coding, how are they going to have the required knowledge if you skip the design and just hand them the finished code?

My experiences has been

3. The business cannot figure out a direction so the developer can either churn on design docs fruitlessly or make prototypes that visually show the business people what our options are in order for them to make up their minds

I think in the age of AI coding, these rationales are a bit outdated. And if you think they're not - I'm curious to know why you think so.

Can you explain in which way they are outdated?

> I think in the age of AI coding, these rationales are a bit outdated. And if you think they're not - I'm curious to know why you think so.

Can you share more about how you think AI invalidates these rationales?

The biggest thing AI enables is cheap code.

That means you could choose to try three (or more) genuine implementations and explore their tradeoffs, instead of making three proposals in a document with one recommended (and the other two usually only provided for contrast).

I do think the design is important to keep around - in particular, the constraints, the communication points, schema, tacit things that might not be clear in code. I am not certain that the design should precede the implementation for features below a certain size though.

Larger efforts need milestones and collaboration and will have multiple people doing implementation, so there's more need to agree schemas, APIs etc up front there.

> The biggest thing AI enables is cheap code.

Agree, but in my experience that doesn't change much about the design doc.

I think it's helpful to the author to be able to say to an AI agent, "Hey, put together this quick prototype," and that informs the design doc. But if the goal is to review the design decisions with the team, I don't see how you get around the design doc. I don't want a teammate to send me 10 KLOC of AI-generated code and ask me to review the design. Even if you told AI to try 10 different ideas and pick the best, I don't trust AI to make the same decisions as my human teammates.

I'm not suggesting using AI generated code as a proposed design.

I would try and get an understanding of design space by giving a good agent a high level goal and seeing what it does, then getting a summary of the approach.

When you do this several times, especially if you give it a steer on some non-functional requirement, you can compare and contrast different approaches.

The idea isn't to prototype so much as to gather information by doing. Prototype, to my mind, suggests other things; shortcuts, stubs, incompleteness. I would actually ask agents to do the whole thing, and find out the full scope. It can be particularly useful revealing side effects.

Pair it with code auditors wearing different hats, of course.

Why do you need to review design decisions with a team anymore?

I get the impression that Fable, when well directed, is better than maybe 80% of SWEs. Maybe more.

[edit: Yes, I'm maybe baiting other users, but I want to know your honest opinions on this.]

I find that LLMs are still worse than humans at limiting complexity, which is one of the most important outcomes of a design review.

If I tell a senior SWE that I'm creating a Discourse-like discussion forum, and I want users to have three options for selecting an avatar: (1) import from Gravatar, (2) upload a JPG or SVG or PNG or GIF, or (3) let the user draw their avatar on a canvas, the LLM will happily go and design that and write a 5 KLOC implementation, whereas a good SWE would push back and say, "That's like 10x the complexity of just allowing JPGs. How about we simplify it to say that in v1, the only option is to upload a JPG."

I've tried working with Fable/Sol and saying, "Look for features that we can simplify to reduce complexity," and they don't understand. They'll guess at features we can cut entirely, but they fail to see how to capture the essence of the feature without the complexity.

I've noticed this a lot with Fable recently. Like I'll say, "Show an error message in the web UI if X fails," and Fable comes back with this like 800 LOC error message generator that has switch-cases and combines inputs from three different sources when all I wanted was something like, "Update failed: database is locked."

I do agree with you put I would push a little further - is it that complexity itself is the enemy? Or is it that the secondary outcomes of complexity (bugs, more effort to make changes, confusing code) are the enemy? If it is the secondary outcomes that are the enemy, and AI actually effectively allows you to mitigate those outcomes (debatable! I debate this with myself all the time!), then maybe we should embrace the complexity (or the agent should on our behalf)

> I do agree with you put I would push a little further - is it that complexity itself is the enemy? Or is it that the secondary outcomes of complexity (bugs, more effort to make changes, confusing code) are the enemy?

I agree, but I think we're still a long way away from being able to trust AI to manage all software complexity for us. For one, LLMs frequently get tripped up by their own complexity. But even if the complexity didn't make LLMs more error prone or expensive to run, you still often need a human in the loop to understand what the system does.

I think of it kind of like compilers. Compilers do a good enough job that 99% of developers don't understand code at the bytecode or machine instruction level, but if we lost that last 1% of programmers who understand CPU instructions, we'd be in serious trouble.

LLMs are worse than any competent programmer. Just turning them loose is a terrible idea.

Even if it's cheap, 3 implementations are more expensive than one and then you add the additional task(s) of evaluting them and selecting one to move forward with.

I think we definitely need to have alignment, and documentation to support it. I think this can be at the PRD level, mostly.

For many systems, I'd argue technical documentation to understand how the internals are working can simply be handed off to the robots. Or generated on the fly. And if a requirement on the product level is not met, that can be changed under the hood.

As the other commenter said, "code is cheap" now.

IME when starting a project from scratch, detailed upfront architecture specs are pretty much required to keep LLMs from flailing around too much (unless of course you build another cookie cutter CRUD webpage, those can simply copy paste from the millions of examples on the internet).

In a way it's a return to waterfall, just with faster implementation phases.

In the age of AI coding, code is cheap. Getting the requirements and high-level architecture nailed down is where the hard engineering challenges remain.

Enter... documentation.

Not OP, but I think they're way more essential with AI doing a lot of the coding. The biggest thing that AI, even the frontier models, is not great at is staying on topic and actually finishing a project with reasonable priorities instead of ratholing on insignificant details or claiming it's "finished" when it's half done.

The most important thing that a good design doc does is specify what's in and out of scope. The second most important thing is to precisely define common vocabulary - what are the important concepts in the problem you're solving, and how should they relate to each other? All of that information serves to ground the day-to-day work in what's important. I find myself starting every Claude session with "read this doc and get familiar with the world, then we'll get to work on a part of it".

(The same is true when working with humans, especially but not limited to junior engineers who aren't used to managing a project longer than a week or two. AI coding agents just never grow out of that phase.)

For an entire project? Yes I agree. For a feature or a submodule? I think when you work with claude to develop a plan, it's generally pretty good.

I guess my question stems from being rigid how a design doc should be defined, argued over, and then executed by humans. I think some of the details simply don't matter, and if they do, they often can be changed relatively quickly in order to adhere to the new requirement.

Try the claude-mem plugin.

I think it's more important. AI gets a lot right, but sometimes it gets things wrong. The document might be the only human authored piece of text, and it will help future agents see that something is incorrect in the implementation.

We have debated this a lot in our organization. We are tired of seeing low effort Tech docs that puts the onus on the reader than the writer. I think that the writer should spend at least an order of magnitude of time more than the reader. If not, then the design doc can just be the LLM prompt that generated the document.

I have actually resorted back to hand crafting TDDs and focusing on 1-2 page docs. It is a great way to organize my thoughts and create a shared mind space among other engineers. My 2 cents.

Can you summarize your thoughts into a single prompt that, with the context of the codebase, gets expanded to something that makes sense?

I agree that we should basically be requiring hand-written-only design docs, because it should force people to make sure they know what they're getting someone else to read. But there's two problems I run into:

1) A lot of people who write design docs, RFCs, etc, don't write them well. I end up needing to get them on a call and explain their entire idea to me because it's the only way to pull the details out of them.

2) Regardless of how much I write by hand, I still have engineers who are so incredibly lazy that they just don't read the docs at all. They can't be arsed. So I have to get on a call and basically explain the whole doc to them.

This is starting to lead me back to what other people hate: meat puppeting. Telling Claude my idea, Claude writes it up, and I ask that engineer to ask their Claude to read my Claude output and summarize it for them. I really want a better solution, but our engineering management is almost nonexistent, so nobody does anything they don't feel like doing.

Reply on news.ycombinator.com