The Surprising Power of Documentation
vadimkravcenko.com
vadimkravcenko.com
You should write three types of documentation. One for users, one for admins and one about architecture.
User docs are simple. How do I use it. What are the API calls, etc. Admin docs are about how to install/break-fix/troubleshoot issues that are beyond user interaction. Architecture is how the system is constructed, why certain tech was chosen, etc.
There's nothing that makes documentation more useless than when you're trying to do something like install the software but you have to dig thru piles of docs about why Postgres was chosen over MySQL. If your users or admins cant find the info they need quickly, they'll soon discard the documentation and user/system ops will go back to word of mouth knowledgeable.
I really think good companies focus on this and those who are successful really shine.
Ex: I work in the energy sector at the moment; because their gas/electricity usage varies across the year, there's a system in place where they pay a fixed amount per month, then they pay or get paid back the difference by the end of a contract year.
It's in the energy company's best interest that the monthly amount they pay is on par. The process to adjust this monthly amount is administrative, but in addition to that there's a huge stack of things to deal with across platforms; website, apps, back-end, support, support user interface, etc etc etc.
All the processes involved in just this aspect of the company need to be documented and drawn out as well, because else it has to be figured out from code or various people that happen to have it in their head. That's where a lot of the meeting culture comes from, because there's no one person in charge of this process, and the ones that know enough don't get together to write it down (and then maintain that documentation).
The reason why is because it allows a diagram to support more of the cross-cutting concerns by drawing a red arrow or a green arrow through things instead of a blue one.
It's a big moment in my realizations around technical communication. Like, I already knew this stuff mattered, but it's made more concrete when you can look at a diagram and the holistic whole helps you make more sense of things even if the details aren't right yet.
I use an airplane analogy (different order than your three above):
1. "Congratulations on purchasing your 747"
2. "This is how you replace the auxiliary power unit"
3. "This is how you survive the engine catching fire"
Edit: word choice
If I'm properly getting your point, you think there should be user manuals, administration and maintenance procedures, and architecture specifications and/or decision records.
Except...
> User docs are simple.
A complete reference of how to do A, ..., X, Y, and Z but lacking conceptual context could actually be detrimental to a customer's productivity and the ultimate success of your product.
Providing accessible conceptual guidance can be very challenging depending on the domain.
If the user needs complex docs to perform a complex task - that's 100% OK. Just don't write complex content when a simple (or direct) explanation will do.
My (perhaps overly simplistic) take would be that we should take the thinking we use on the product itself (Who's going to use it? In what context? What would they already know? And so on), and apply and adapt it to the docs as we would any other product.
mostly the same but some additional information for people who are interested
Technical writers train specifically to communicate complex technical topics to readers, and it's not an easy job. It requires understanding your readers, what kind of backgrounds they have, and what are they trying to achieve. This becomes especially important for documentation that is meant for your customers, where very real revenue depends on the quality of your docs.
I'm a bit biased since I'm the founder of a documentation startup [0], but tools also do play a big part. Devs often tend to enjoy writing something Markdown next to their code than going to an old wiki like Confluence that's disconnected from the engineering cycle. Choosing the right tool lowers the barrier to keeping the docs up to date.
Great phrase!
To me, there are three places that dev-generated documentation can live:
1. The code
2. The issue tracker
3. The version control system
A small amount of exceptionally useful and frequently referred to documentation like the process for setting up a new dev environment or some complex support task can live elsewhere.
Otherwise, I think the top down imposition of a documentation culture is unlikely to succeed.
The real secret to getting a team that has a shared understanding of the system, the business, and each other is to retain your developers. A team that's been together for five years has superpowers no amount of documentation can replicate.
I've worked in multiple* companies where the problem was too much documentation, and of course everyone was afraid to update or ghasps* remove any piece of old documentation in case it was still useful. Imagine working on a codebase where 80% of the code was unused or commented out but no one dared changing it just in case (flashback to 2010 with 4000 lines of style.css).
I'd suggest to take a more holistic approach and treat documentation a lot like testing; for that prototype, probably just write the barebones documentation, for the production-ready new feature go all-in and write detailed documentation, tutorials, etc.
If you do want to go deeper with documentation, then you'll need a dedicated team (like a team of testers) that work exclusively on documentation. At some point it does make sense to hire only for that, and it can even be a differentiating point for your startup if done correctly.
For libraries, a ratio I've seen works pretty well is approx 1:3:5 for lines of code:tests:docs; you can do tests first, or even documentation first, but once everything is finished and if you count the amount of lines that's a decent ratio. Note that when counting "lines of docs" in an editor, a whole paragraph will count as just 1, so in reality there's a lot more docs.
Note: I'm the creator of both https://documentation.page/ and https://documentation.agency/
* (only 2 "negative" paragraphs on a book-length article)
Even better: if you have examples in code blocks in these docstrings per default they get tested as well, so if you don't update them, the tests will fail and you will notice.
In my eyes one of the biggest problems with keeping documentation up to date is that over time the mapping between the piece of code you are documenting and the place where you find it in the documentation becomes more complex, to a point where missing something is not unlikely. Rust's documentation-in-code-approach addresses this problem neatly.
Then as the code grows, you need to document the architecture, add small gotchas, etc.
At the end of the day, documentation wins.
get_height gets which height, outer or inner? Are there error values, e.g. 0 as "don't know any height"? Does it have side effects? Is it a stable and reliable part of the API or bound to change soon? Is it thread safe? Will it change any of its parameters? Who deallocates the return value? Do you need to hold a lock somwhere?
Of course it might be a good idea to group together get_height, get_width, get_diagonal and get_depth if the above is all the same for those. But having no documentation just because you think it is trivial that get_height gets some height from somewhere just means that you are sloppy and didn't think of all of the above. So your code shouldn't be touched with a 10-foot-pole imho.
My solution, which I personally hate but know of no alternative to, is a documentation template for each function asking the above questions (depending on runtime and language of course) that I give people to fill in. Until they learn...
And, yes, I don't like this either and would like a better solution. But so far I didn't get any viable suggestions.
const fn get_outer_height() -> Result<SomeErrorType, WeakReference<Number>>
- `const` makes it clear this doesn't mutate
- the function name says exactly what it does
- The return type makes it clear it can return an error
- The return value is typed in a way that makes it clear what the ownership is
Throw in a language like Rust that gives guarantees about thread safety and now the only thing left is if the API is stable or not. Which I would argue doesn't matter much at all since people will still end up depending on it regardless of the comment saying "This API might not be stable"
And the best part? My definition will never get outdated. If the assumptions change, the definition will also need to change (well, except for maybe the name)
But often one doesn't have a choice. People still write software in inferior languages such as Javascript or Python, where you cannot even be sure about a return or parameter data type.
Quite often something is not obvious or trivial to someone who is examining a piece of code or using a library for the first time because it assumes the person already understands the context.
An example of what I mean: perhaps it is because I have a background in the sciences, but I assume that most properties have units. A property such as height certainly does have units. So is get_height() returning the height in pixels, inches, meters, or something else? I have also been bitten by graphics libraries that measure distances in unexpected (to me) way. Is the radius of the arc with line thickness 'n' using the inside radius, outside radius, center line, or something else? The person writing the original code may think the developer using their code down the road can test different assumptions, yet the reality is the number of combinations to test will rarely be trivial (and that is assuming they identify the correct parameters to test).
It's at the point where I refuse to even consider using libraries that leave out documentation for obvious things. Even comments like "gets the height" raises red flags since it is a demonstration that the author did not put any thought into what they are documenting.
In my experience, if your documentation covers all these aspects it's guaranteed to be either wrong, misleading, or out of date on any of these details, and you better read the actual code to be sure.
In particular, the answer will often be "it depends on what the rest of the system does". Perhaps it delegates the actual calculation to an API, and this doc won't change when the API changes in a _mostly_ compatible way.
I mean, even with the best efforts given, code has bugs, words are vague, there's no way you should trust the dev who wrote the code to properly convey what it actually does.
Like you say, get_height is a trivial function, but still requires attention. Enforcing in-code docs is not going to help to have higher quality docs, quite the contrary. You often get low effort stuff, just to make the code checker happy.
And if you put in a peer review process to validate the in-code comment, it loses all power because you might as well use that step for decent documentation. get_height should be documented in a logical place, where it makes sense indeed, like grouped with get_width. But now you have the logical place to put your documentation, and the forced javadoc comment. That's double work, and one of them will be bad quality as a result of it.
Nobody is arguing for no documentation, but I am arguing for avoiding javadoc enforcements. My solution is much simpler: have a documentation check as part of the peer review process. Sure, have a template for the documentation, but don't make it strict. Ours is simple: all juniors are on documentation peer review as part of their onboarding. If they don't get it, it needs to be fixed.
Our peer review process is quite simple: is the documentation adapted, is there a relevant unit test (we actually have low UT coverage, we only do them for critical code and as part of bug fixing, so enforced frameworks make no sense for us), and naturally is the code quality itself ok
But many companies don't do those, yeah well thats how you get shit.
You say "gets the height" is trivial, but that text tells not only what the function does, but also that the author couldn't think of anything else important to say. This is very different from no text at all, where you can't be certain if the author even thought about it.
IMO, enforcing an internal structure (AKA "you must document each parameter and the return value") is counter-productive, but enforcing the existence of the comment is very productive.
> Also it clutters the code with many highly trivial remarks.
I'd say that it "clutters" the code with markers for public elements and hard to understand ones. Those are actually valuable, and not clutter at all.
Anyway, if those markers are a large share of your lines, you may need to rethink your architecture. It's usually not valuable to have a lot of interface for trivial things.
Also, I don't tend to trust it as much, because it isn't actually what the program executes.
When I'm trying to read the source code I don't want to read about the source code -- I want to read the actual source code -- and if I keep coming across long multi-line idiotic comments then it breaks my flow and concentration.
I like the source code itself to be extraordinarily readable, with long and descriptive variable and method names, but I want it to be dense and packed into paragraphs of sense.
To the extent there are any comments at all they should be extremely short and completely clarifying -- they should not even partially overlap with information conveyed through function or variable names, for instance.
Maybe you don't know Rust, but it is a strongly typed language. If you go to a rust project and run `cargo doc --open` you will see exactly what the type system lays out. Sure if someone writes "changes the windows height and returns a new Window" on a change_window_height(height: Pixels) -> Window and it changes the windows width instead that is still wrong. But (A) you can click on "source" and see the actual code and (B) you are guaranteed that the function takes Pixels and spits out a Window
That being said I hate that kind of documentation in most other languages, not in Rust. In Rust it is basically an alternative view onto the same code with a (needed) emphasis on the realtions of the entities the type system describes. If there is prose that can be a nice extra, but cargo doc is even useful without a single comment.
Bridging the gap between a prosaic high level explaination (how do the parts work together?) and a fine grained explaination of each part (what does that part do?) can be a challenge. Rust solves this somewhat by allowing you to do module level documentation (that can essentially look like a blog post, only that the examples get checked when the code is tested) and it lets you link to different entites.
I would always prefer a well written blog post, if people were able to keep the examples working and the code up to date. But experience shows they are not.
I'd rather have generated documentation that is true than a blog post where half of the examples won't work, because nobody bothered to update the post after the code changed. The first might at times be barren if done badly, the latter is downright misleading.
If you write tests in a strongly typed, non-turing complete markup (in this case, StrictYAML), you can then use it with a template and test artefacts (e.g. app screenshots) to generate readable how-to/tutorial docs which are guaranteed to stay up to date.
https://github.com/hitchdev/hitchstory
This isn't a new idea, but I find that people are often skeptical because there's a history of people getting their fingers burned by Gherkin's language design or YAML's weak typing (both of which are completely valid).
Conceptual documentation, the big picture, is important to convey the mental model implemented by an API. In applications, you can generally infer it from using the app, but it's not always easy, and it's indirect.
Use case examples string together multiple APIs, multiple domain objects, to achieve a high level business objective. When working on a project, you can sometimes get away without this - the existing code can be example enough to copy. You can end up with cargo culting, people copying things without understanding why. But if you have an API for third-party use, which needs documenting, you need to have either a well-seeded set of open source users, or a great set of examples.
See the Clap documentation:
That must be nice. I've yet to work on a company where half my time wasn't trying to prod for some resource (be it internal code, a public 3rd party tool, or even the resource itself), sometimes playing a game of goose just to figure out who knows the author. I'd love too much documentation.
But I understand your point. The only thing worse than no documentation is wrong documentation, and outdated docs half the time can become outright wrong half the time, if it isn't simply encouraging outdated but functional practices. Tech writers are highly undervalued for that purpose.
I should also mention that the ability to properly search for docs is almost more important than the doc itself. Some companies had wikis but good luck searching for the right keywords if you didn't know the exact title. A properly categorized top level page could have helped a lot (and is probably easier/cheaper than integrating google like searchabilty into an internal database).
> The only thing worse than no documentation is wrong documentation
Yeah, that was the exact problem. One was a startup and when I joined it was all mostly up-to-date so it was great! But by the time I left (after 2 major migrations) most of it was out of date and a nightmare to find anything updated, any script that could still be run, etc.
Documentation is such a hard problem to "solve" if you're a fast-moving startup. You need a mixture of creating a documentation-first culture and acknowledging that documentation is difficult to maintain. Ultimately you end up creating processes to help people document their intent, decisions and the mission critical information.
There is also such a large range of types of documentation - varying scale from internal to external and technical to non-technical.
We started by creating https://writer.mintlify.com/ which really resonated with developers because it made it easier to write documentation, but it only helped generate documentation that was highly technical and close to the code. We decided to stay in the documentation space but try another vector and so now we're taking a crack at public-facing documentation - which in my opinion is a different can of worms than internal documentation. However as I'm building and growing my startup and I find myself continuously playing whack-a-mole and I definitely hope that we can build the foundation and expand to make it easier to maintain all different types of documentation.
A tiny bit of feedback: the shortcut key is defined in settings, not keybindings which seems wrong? Also, the default is to override cmd+. which is an important shortcut already..
That's not documentation; that's code.
It's an analogy to make the point - we delete old code that's not relevant any more. Imagine reading documentation where 80 percent of it is no longer relevant but is kept around "just in case".
The key word is "good" documentation. That takes time and effort to write, and it takes time and effort to keep it updated as things change. As the author notes, it will have to be something that is made part of the culture of the organization. And given that recent agile programming approaches proclaim that "the code is the documentation" and that formal, separate documentation is an impediment to productivity, you'll find that many developers will dig in their heels if asked to write documentation.
Bad, outdated, or just plain wrong documentation can be worse than nothing, as it tends to lead you to incorrect conclusions and beliefs about the system.
To write good documentation you need to mix technical reference (the easy part) with user reference. The latter requires you to imagine where the user is at, and take them to where they understand. This is hard to do, and requires well, skills.
So a culture of documentation is great, but quality matters as much as quantity. Clarity, completeness and coherence are all legs of the stool.
However IMO an easy trap to fall into is to start documenting without a bigger understanding of the audience and the purpose of the doc.
A good way to start is to identify which of the 4 types of documentation you are working on.
https://nick.groenen.me/posts/the-4-types-of-technical-docum...
Personally I find it very easy to put too much explanation in the wrong places.
If you are fortunate, you can write code in an organization which has a high code quality bar, uses consistent styles etc. But it is rare (vanishingly so I nearly 30 years experience) to find the same bar applied to the design docs.
Then a bunch more degrees that I could know about if the tool were more usable
I mostly use man pages when I already basically know the program and need to do specific thing.
Even then I don't use man itself but usually a webpage of the manfile because of UX.
OpenBDS's developer to user ratio must be less that 1 to 1000s. When you update one line of document, you're probably saving time for thousands of users accross years of use.
Most project I worked on had at most a few dozen people with an actual chance of reading the documentation, and the majority of them aren't users, they'll be reading all the code anyway because they're not in a position to blindly trust the documentaton.
Why should we calculate the ROI of the time spend on maintaining good documentation the same way in both cases ?
PS: I also think a distinction should be made between specification and documentation. It feels that both are conflated too many times.
Much like automation, the question should be phrased in terms of how long it will take and how much time it will save. https://xkcd.com/1205/
Other things that I see treated as documentation when they're not: slack messages, uncommented code ("self-documenting" code exists, but it's much rarer than management seems to insist), vague jira tickets, some guy who worked on the app 5 years ago and is happy to answer questions even though he's in a new role now, etc.
I've often seen developers who spend hours fiddling on some detail that was clearly mentioned in the readme of the very same repository containing the code on which they are stuck, or who just failed to read the extensive documentation and proceed to 1) run the code and 2) call me for help. Furthermore, I've actually caught myself doing the same more than once.
This led me to think that for a good documentation culture, the primary question should be: how are developers actually going to use and benefit from the docs? How documentation will get updated is the second question of importance, and writing documentation comes third.
This also makes me bad myself at documentation, because if I don't use it I also feel internally that no one would read what I write in the first place also.
Out of responsibility I will try to document shared things, but I never feel productive while doing that, I feel like I am just writing into a void.
There's another big issue with documentation; it's often a write-and-forget thing. I'm confident every team or department should have a full-time documentation owner whose job it is to ensure documentation is up to date, maintained, and verified.
Short of that you'll need to explain why the company is losing money because Jim didn't write a full explanation on why his "getUserIdOrNull" function returns null when the user id is not available.
I'm not convinced this would work. Such a person wouldn't have time to be a subject matter expert at anything other than the documentation tools. They wouldn't understand what they were writing about.
I usually see software build by brilliant engineers, people much smarter than I at building software, and the documentation sucks.
Not that is not there, but that is very difficult to understand.
It is clear to me that you can be very smart at doing certain things (for example writing software) and suck at others (writing documentation).
Now we're stuck with both a chicken-and-egg problem at most places, while in places with decent documentation, many developers still come in not reading it, discouraging any significant upkeep of existing documentation.
If people expect that the docs won't be comprehensive or will be out-of-date, they won't use them.
This is why I'm so keen on documentation living in the same repo as the rest of the project. That way it can be kept up-to-date with the state of the code, through a policy where PRs are only merged when they include the relevant documentation updates.
Having that policy in place really helps people learn to both write and read the documentation.
It helps a bit, but only if your code reviewers are actually going to enforce the rule. It seems like most programmers simply don't like reading or writing documentation even though they can save large amounts of time for everyone by doing so. Certainly it's frustrating to be told e.g. "this thing we use can't do X" when how to do X is discussed in the thing's user guide, simply because someone didn't want to read it. I've had that experience before.
Code reviews let you force the issue for a while but it's hard to scale. Getting other people to enforce the same rules via review is difficult. Many devs will be really strict about things like unit testing and make it a point of pride, but not at all strict at all about updating docs. Other devs will learn which reviewers let them avoid writing things in English and send reviews there preferentially, or find other ways to dodge it. Fixing this via training or policy turned out to be nigh-on impossible: many people simply will not change regardless of how many times you send a code review back for missing docs. Nor will they change even when their questions are constantly being answered by a link to the docs, which they don't take as "you should be embarrassed that I had to google that for you" but rather "hey here's a helpful link, you're welcome".
Fundamentally there's a lack of shame about not reading things. It's not unique to software either. People ask questions answered by docs, or even by emails they just received, or they make bold assertions contradicted by docs they claim they've read, and when this is pointed out they just shrug it off in a way they wouldn't do if an obvious bug snuck through that should have been caught by testing. It's a cultural issue and needs to change, really.
One fix I'm experimenting with at the moment is for a Linux kernel style hierarchy of reviewers where everyone gets their own repo and they merge upwards, so there's at least clear ownership and if someone is consistently letting docs rot that becomes apparent to the ultimate TL when they do a quick eyeball of big merges.
It may also be worth experimenting with large language models. They could be given a change and asked, "given policy X, should this change have updated the documentation?" and if the LLM says yes then that gets flagged centrally for followup, for example.
It helps a bit, but only if your code reviewers are actually going to enforce the rule. It seems like most programmers simply don't like reading or writing documentation even though they can save large amounts of time for everyone by doing so. Certainly it's frustrating to be told e.g. "this thing we use can't do X" when how to do X is discussed in the thing's user guide, simply because someone didn't want to read it. I've had that experience before.
Code reviews let you force the issue for a while but it's hard to scale. Getting other people to enforce the same rules via review is difficult. Many devs will be really strict about things like unit testing and make it a point of pride, but not at all strict at all about updating docs. Other devs will learn which reviewers let them avoid writing things in English and send reviews there preferentially, or find other ways to dodge it. Fixing this via training or policy turned out to be nigh-on impossible: many people simply will not change regardless of how many times you send a code review back for missing docs. Nor will they change even when their questions are constantly being answered by a link to the docs, which they don't take as "you should be embarrassed that I had to google that for you" but rather "hey here's a helpful link, you're welcome".
Fundamentally there's a lack of shame about not reading things. It's not unique to the software world either. People ask questions answered by docs, or even by emails they just received, or they make bold assertions contradicted by things they claim they read that morning, and when this is pointed out they just shrug it off in a way they wouldn't do if an obvious bug snuck through that should have been caught by testing. It's a cultural issue and needs to change, really.
It's possible AI can help here. Some people just don't want to sit down and read, but they'll ask questions, so an AI that reads the docs regularly could motivate people to write it. Or you could use them for code review enforcement by giving them changes and asking, "given policy X, should this change have updated the documentation?" and if the LLM says yes then that gets flagged centrally for followup, for example.
I had similar policies. Also periodic reviews and especially during onboarding new devs I had them review the docs and make a list of missing points etc
Its just as important to delete out of date topics. Wrong and stale info is costly, it decreases trust and sends people in wrong directions.
edit: another obvious but surprisingly difficult thing to do is have everything in one system and make sure everything knows where to find it. Especially if your org is a bit bigger everybody will have their own systems and it'll be a big mess before you know it. But even so, within a single project I've often seen 3 to 8 'sources of truth', like trello tickets, images in onedrive, a word doc, markdown files, various propietary formats, pinnend posts in chat system, email archives, etc, etc.
But what parameters are available for the automation? Where does the automation live? How do you diagnose and improve when the automation breaks? Why did we even make this automation in the first place?
These sorts of questions are ripe for documentation. Most How style questions can be automated in one form or another. But the business process behind the automation, the context and domain knowledge around the autmation, for the humans who did not personally code it, documentation has major benefits that I think we as an industry don't value enough.
A case of non-automatic automation :)
Actually, there's nothing wrong with that. Many things are best left that way. It's just an interesting oxymoron. And it's also interesting the fact that yours (and I'm sure many other's) mind jumped directly into it.
Automatic automations also exist. And those require a complete different set of documents.
But anyway, they are important because the "how do they work", "how do we fix (or improve) it", and "what can they do" are trivial to deduce from a working artifact. Those are not questions you usually want to answer with text.
If there is no documentation for the system architecture or how to solve common problems, everyone on the team wastes cycles solving problems that others already have, and doing it in different ways, so the codebase becomes an inconsistent pile.
Documentation is truly a force multiplier. It allows an entire team to learn from the experience of a single person, and that person can help the others passively and asynchronously.
Is there an example that actually happens? A system requiring 100 pages of documentation just for installing is not getting done in a single script
"You can think of Documentation as essentially the backbone of effective knowledge sharing."
"In the words of Bukowski, 'Don't do it unless it comes out of your soul like a rocket', apply the same principle to meetings."
"The constant need to have meetings is a symptom of a deeper problem — a lack of clear, accessible, and reliable documentation."
"Encourage your team to document their decision-making process to clarify assumptions, reasoning, and expected outcomes. Make it a standard practice to discuss these documented decisions in your meetings, promoting a culture of open feedback and collaborative decision-making."
I've been working in startups for several years now at companies of a variety of sizes, all of which were remote-first, and which (ostensibly) relied on writing to communicate.
People do not read what you write. I don't know if they can't actually read fluently or if they won't, but it does not matter if I submit a bug ticket that says exactly what is happening and lists the ten things I've already tried to resolve it. 100% of the time, the first reply is to ask if I've tried doing any of the first three things I said I already tried.
It's that kind of thing that makes me think documentation is hopeless. Nobody's going to read it anyway.
Then you need to move on, and find different people.
Yes -- I know it's tough. The landscape out there is quite bleak, in fact.
But these places, and these people do exist.
Fuck you. I’ll leave you unread until end of day then send you the docs you clearly didn’t read.
Maybe the problem is that it is treated as a secondary activity for developers, when it should be treated as a primary activity for writers.
We don't expect developers to be good at graphic design and even UI/UX design. In fact we should expect them to be terrible at it. A developer looks at the product from the inside, he sees classes, databases schemas, etc... not the way an end user will look at it. It means he will be biased into having a UI match the code structure and not the user workflow. There is a reason UI/UX designer is a job title, it is not a secondary activity for coders. Some can do both, but it is a different job.
Documentation could be treated the same way. Have people specialized in writing documentation. People who are actually good writers. I have seen it happen occasionally, and let me tell you, when you put a good writer (coding skills optional) in charge of writing documentation, the difference is night and day. Just as how better your UI will be when done by a good UI designer, and by a good UI designer, I mean someone who actually designs for usability, not someone who just tries to make something that looks cool for sales presentation, as it is too often the case for consumer apps today.
To me what's missing in many of these discussions is the cost/result calculation, how much effect is expected from "documentation". Thinking of it as an UX/UI could help put it more in terms of what time is spend by which user to achieve which specific task.
If specific use cases can be described, what needs to be written down becomes a lot more obvious and it can be done way more efficiently than just blindly "documenting" a system.
We’ve built Lowdefy [1] as an open source project and documented it with all effort, 200 pages of docs. I often forget why or how something works and then jump to the docs. This investment keeps on paying of as we use Lowdefy to build customer apps, new devs in the team typically take less than two week to get up to speed and start making contributions, the sharp ones, just a two or three days.
This year, we’re extended our documentation onto customer apps aswell, with flow diagrams, state machine definitions, detailed field level explication schema definitions, and end user test procedures. The key here for this documentation is detail. It should be easier to reach for the docs and the the answer, than to dive in the code and interpret it.
This, given of course that the tech you are picking up has good docs.
As you said, without it, you’re in the dark, doing guesswork. Doing that with multiple people, like a call with everyone guessing, is even worse. Just have everyone read the docs on their own time. So valuable.
(a) it's an ambiguous material (like wood), so anecdata go all ways
(b) it's for the future, i.e., easy to cut in a time pinch
(c) it presumes knowledge is shared, though it's often hoarded
(d) it's a tax on everyone's time
A helpful discussion of documentation would focus on specific use-cases: on-boarding developers, backgrounding design discussions, operational run-books...
In that context
(a) The cost/benefit is concrete
(b) You've identified the consumer/stakeholder, so they can speak the the present value
(c) Present work is value in terms of that future product
Then some documentation strategies become clear:
(1) Write for some specific reader. It's not a brain dump (unless it is, e.g., for departing engineer).
(2) Build in feedback cycles with actual users before completion
(3) Make it someone's job (put them on the hook) to deliver good documentation (for all users). They can optimize extraction and repurposing across the organization.
If I see director+ level people with no strategies for documentation, I conclude they're not building an organization.
We have over 700 routes (screens), 1200+ components and 500+ different service calls that query APIs in the back end. If we only look at routes and hope to spend, on average, one day per screen, that's 700+ days of writing docs, which is a considerable amount of work.
There's no existing documentation, save for 30,000+ JIRA tickets over a 5-year period, that describe various bug fixes and change requests. But those tickets are just floating in the ether and are not formally attached to any specific component, let alone route.
I was hoping AI would help but I can't seem to find anything relevant.
What would you do?
Accept that it's a big job and just get on with it. Sometimes we just have to do hard things. Putting it off or looking for a shortcut doesn't always work.
I'd also spend a couple of months seeing how much of the documentation production I can automate though. That's a small investment in a 700 day project.
> If you have a mountain of shit to move, how much time should you spend looking for a bigger shovel? There's no obviously correct answer - it must depend on the size of the mountain, the availability of large shovels, how quickly you have to move it etc. But the answer absolutely cannot be 100% of your time. At some point you have to shovel some shit.
From https://www.scattered-thoughts.net/writing/things-unlearned/
We intend to have designers update the doc when describing a change request, and the devs also update the doc afterwards.
We're not sure any of this is going to work; in the life of the project there has been at least two major documentation efforts, that failed because they were eventually abandoned (not maintained).
Some are still hoping there's some kind of magic bullet that would let us automate everything... and I'm not immune to this myself.
It sounds like you're thinking about the really boring kind of documentation. The kind that no one wants to read and certainly no one wants to write.
In some approaches, the 4 types of documentation are tutorials, how-to guides, technical reference and explanation.
Any per-screen documentation is not tutorial, how-to or explanation. Perhaps it might be technical reference.
The first question then is why are you working on technical reference? Would you get more bang for the buck writing how-to guides?
What we do not have is something that describes exactly what the system is supposed to do, so that when one stumbles upon an unexpected behavior, they don't know if it's a bug or if it was intended that way (it could be either, depending on old requirements that weren't properly written down).
Not everything requires a lot of documentation, we have a lot of essentially glorified input dialogs, but we do try to write down what it is, and especially any logic and config settings that affect that logic.
One thing I've used before with success and which we've also introduced was two new fields in JIRA, "requires documentation update" and "documentation updated", to aid not forgetting to update documentation when adding or changing code.
and make sure this is at least one person's actual full-time job. chances are you'll have to hire specifically for this role because nobody wants to be bait-and-switched into this job, but there are actual professionals that do this for a living.
Were using LLM retrieval methods to build Q&A bots at work. These are all fed with documents (user guides, release notes, transcribed videos etc).
Its still very much POC but the interesting thing is people seems to care about documents again a bit more knowing that it will be used in this manner.
I was thinking about developing something that rewards document producers if their response is cited and used successfully - would help strengthen the feedback loop.
- it takes effort to write it
- there is a split between what documentation describes and reality
- it falls behind as new stuff develops, project gets forked, taken over...
My solution is to have code examples, that are part of unit tests. Separate folder that describes most common use cases. If documentation is wrong, project does not even compile or test fails. And I can always point to most current version of examples in git branch.
I really think any document beyond simple readme.md is overkill for most projects.
About X Installing and configuring X Using X X Reference
Typically developer docs are created from the bottom up. The devs create the preliminary reference docs using special comments in their code.
Once they are through, I go through their comments and wordsmith them.
After the reference topics are written, I start adding a "guide" section. I like to call this the "How-to" section, which answers questions like: * How do I create Y * How can I ... and so on.
I try to answer two classes of questions: * Tasks that everyone does (create a client, ...) * Tasks that flummox a lot of people (talk to the folks manning the help desk)
Once I'm happy with these task-based topics, I'll create a simple "Hello world" tutorial. This topic helps the user know that they've successfully installed and configured the software.
Finally I'll write the installation and configuration section.
It's possible to work on more than one section at a time. In fact, I typically write a bunch of sample code to try out ideas before I create the guide and tutorial. If possible, I'll tidy up these code snippets and add them to the docs. Developers always ask for more code examples.
And speaking of which, if you do create a code example, please create an accompanying unit test. Don't make your users find out that version 1.1 broke your code example. That's your job.
doug in Seattle
According to this saying, the fix to stale documentation is (often implied) to not write documentation. Can't go stale if it doesn't exist!
The above saying is so often used as an excuse to write no documentation so much that while stale documentation can provide more acute pain than no documentation, the chronic pain of no documentation is a cure worse than the disease.
Companies in general should do much more writing. Writing forces you to think in ways that coding doesn't. For me it's much easier to spot a poorly thought out argument then a bug in code (not a 1 for 1 comparison).
“Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.”
— Fred Brooks
With the mental model of the software, I know where to go, where to look, how to change to fulfil my new requirements.
I am thinking of writing a fictional documentation for a fictional operating system or library or web framework and then see where that design takes me.
With good documentation, it can be used to scale yourself beyond what you can personally do everyday, and it works really well when you can convince people to search for answers before asking
To identify the hot topics on support tickets, the docs team can liaise with the customer support team to get the support tickets data and figure out the content strategy. The docs team can add sections like FAQ, troubleshooting, customization, and best practices to address the queries. This can yield in reducing future support tickets.
You also need help from the support team on this task. When a customer raises a query that is already present in the docs, they should share the response along with the respective docs page link. This action would help the customers identify that the docs page is up-to-date and their queries can be resolved through self-service rather than a support ticket.
Most of the time, this means making things smaller and modularise those. It takes a lot of work and there is always an initial resistance from the team for doing it. But it doesn't take much for them to realise how powerful and useful those interfaces are once they are in place and work.
A paper alone with code is kind of dumb, you need something people can interact with by discovery, by doing, experimenting, just like we do as children with the world surrounding us.
Just having some explanation is not enough: People just don't understand things reading about them, but formulating hypothesis about their understanding and confronting those with reality.
Without them, people are not going to be understanding what you believe they are understanding, but their own idea, that is often totally wrong.
That's not easy to achieve and takes time and resources to get right. That's primarly why so many fail or give up on it.
The results can very much be worth the effort, however, the ones who should be responsible for the documenting process likely don't see its importance. From their perspective, what they've worked on is easy to understand and requires little to no explanation. Taking time to change this mindset and create proper documentation is an effort many are unwilling to take.
Given that Bukowski said that about writing, it applies more to writing documentation, than to holding meetings. So exactly the other way around than presented in this article.
Some of my stuff is pretty sprawling, I've started integrating the documentation with the code and basically use readme.md's littered in the code as sign-posts to let you navigate it more quickly. The intent of that documentation is pretty clear, and the shape follows logically.
e.g. https://github.com/MarginaliaSearch/MarginaliaSearch/tree/ma...
“ Designating a dedicated team or individual for documentation in an early-stage startup can seem extravagant. But trust me, it’s one of the smartest investments you can make. Why? Because knowledge is the lifeblood of your startup, and a dedicated handbook team acts as the circulatory system, ensuring that this vital knowledge flows freely and efficiently throughout the organization.”
So many startups lack technical writers, let alone docs teams. Not even OpenAI has one, as far as I know.
Good reminder.
I have a bit of a screed on the topic, that I did, a while back: https://littlegreenviper.com/miscellany/leaving-a-legacy/
Second, keep it plain and succinct. No convolution. No wordwalls.
Third. Use pictures and diagrams whenever possible.
(From my father, a technical writer of some renown.)