The tree-based approach to organizing documentation sucks
worldofbs.com
worldofbs.com
In contrast, your API or your frontend code or your Amazon Lambda or your Microservice is quite likely not feature-complete, does some things that should be handled by a different component and was developed with exactly one use case in mind until it was "good enough". Thanks to scrum, no one cares about completeness, orthogonal design, or composition of smaller parts anymore. Hence documentation has only token value. Except, maybe, for end user documentation, but I am yet to encounter a "user story" that begins with "As a user, I want to do X, read how to do X in the documentation, follow the instructions and get the desired results."
A developer's compensation is based upon their perceived value (how much it seems like they are needed) and replacement value (how much would it cost to replace them).
A developer that answers questions gets a reputation for helpfulness - this increases their perceived value. Another person needing to go to them to ask questions increases their replacement value too - the more they become the go to person for info, the clearer it is that the company would lose out if it replaced them.
Written docs have almost the opposite effect. Writing amazing documentation doesn't tend to get you a reputation - mostly because it's so depersonalized (does anybody think about the person who wrote a quickstart guide?). It also decreases your replacement value - a fantastically documented project is a project that can be easily handed off to someone else.
I really think that humans get a buzz out of being helpful, too, and you just don't get the feedback that gives you that buzz by writing pages in confluence or whatever. You will likely not even know who found it useful, whereas the dopamine hit you get when somebody asks you a question and you answer it and they click the "heart" button on your slack message is instantaneous.
Another possibility is that you make yourself entirely removable or even just solve the problem so well that it no longer needs headcount. Instead of seeing people reaching out to you as satisfying a "I feel needed" emotion, you can see the need to have been reached out to as a tax that reduces the value you create.
Businesses really like developers who reduce replacement costs, solve their problems, and enable others. They tend to put them on increasingly valuable and interesting problems.
* They're not very good developers and not very smart and probably somewhat cognizant of that.
* They're afraid of interviewing.
* They're working in smaller markets (e.g. a town which has maybe just one or two companies that hire developers).
* They're spending right up against their income or beyond it and are afraid of losing their job.
If they hit all of these checkboxes then I reckon this behavior is almost guaranteed and getting potentially "redundancy inducing information" out of them will be like pulling teeth. They'll be busy a lot. Far too busy to talk to you. Their docs likely won't be nonexistent (too obvious), but they will probably be deliberately obtuse.
They're primarily concerned not with learning but with not losing their primary source of income and their time horizon shortens, much like it does for those stuck in a behavioral poverty trap https://www.sciencedirect.com/science/article/abs/pii/S03043...
Yes, this sort of behavior locks you further and further into the role you are in and is possibly self defeating, but it's entirely natural and fairly common.
When you've been burned by stale documentation (or any tool) enough times, you can start to lose trust in it. When you lose trust in documentation, you second-guess it and often pursue a a live, fresh second opinion from another developer with more knowledge anyway.
Another large problem is developers not knowing what they're looking for. Documentation tends to solve the "how I do use this" question and not the "do we already have a solution for this" question.
I'm still not sure how to solve the problem of discovery outside of telling people, "read the docs and hope that someday you'll remember the relevant bits when you need to."
The reality is, this sort of behaviour isn't penalised or shamed. Said developer got called out in a meeting with many others present and didn't even appear to feel embarrassment or apologise. Makes it tough to get people to write and take pride in documentation when that sort of thing happens.
I would not want to work with anyone who thinks public shaming is right or even effective. It kills morale, makes people fear asking questions, creates a low-trust environment, and frankly is just plain rude.
In the noted case there was no overt penalizing (at least that we heard about) nor was there shaming (the person was apparently "unshameable"). Therefore your post falls short of it's target.
That said, in general, I agree with every word you said. I especially like the "frankly [it] is just plain rude." - good manners (etiquette) can make a world of difference in a workplace but have largely fallen by the wayside in our culture.
I feel the best courses of action are to remind the developer privately and/or to make a general "remember to read the docs" statement to the whole team in the meeting.
Yes, but we don't know what the OP meant by "called out": it could have been as simple as one person asking "Did you read the documentation?" and a long following uneasy silence, or it could have been quite vicious, e.g., the documentation's author ranting at the developer for not reading the docs, a manager dressing him down for his failure to do so, or even worse.
But we don't know, so while your point is well-made it is tangential to the discussion. Nonetheless you reveal a well-developed attitude about people and groups that would be very useful in management.
As an aside, the OP's description of the developer in question paints an individual who is most likely not as self-aware as you are: indeed he/she sounds like someone who is quite hardened and aggressive in his ways. Sometimes you must unfortunately pop such people on the nose to get their attention.
Compare yourself with the aforementioned developer - which of the two is more likely to be/become a manager? Such an opportunity is not likely for him but is something that probably lies in your future, waiting, if you haven't already gone there.
new folks will show up and start complaining that "stuff doesn't work", at which point I will ask them to read the many READMEs that been sitting right in front of them.
Later, they will say "this documentation is wrong", where inevitably someone else changed some code but didn't change the documentation that is colocated with it.
Because the docs were wrong, nobody will read them next time. And the cycle continues.
But have you ever had a salesperson at the pub tell you about how much easier it is to close deals by pointing to the API integration tutorial you've written?
Hands down, absolutely the proudest accomplishment of my career.
I'd say that, in general, people don't like writing docs or answering to people. But if someone asks you a question, it'd be very hard to decline to answer, whereas writing documentation takes effort and initiative (and it won't prevent people from asking things, because most people don't like reading docs). Unless there's someone pressing for people to do it, they simply won't. Both because they don't care about doing it and also because the activity is not valued anyway (ask any project owner if they'd rather have great documentation or one more feature).
Also, the value of documentation is questionable in the simple software using standard components as most of us do. I'd be interested in seeing a study to know whether good software docs really improve productivity or reduce the number of bugs.
However even if you take out the replaceability I still think that there's a relatively limited upside.
Thus you're left with conscience or a sense of professional pride as motivators to write docs.
My general approach is: if a question gets answered in chat, also add it to docs. Personally I dump each question as a markdown header + paragraphs (typically with an error message or uniquely identifiable string in the header), using a FAQ format under whichever page makes the most sense. Then it's easy to find the section w/ the search widget (we use js-search on a next.js SSG site) and it's easy to link to the specific header. Then, the next time the question gets posted on chat, just open docs, search, copy permalink, paste, done.
I agree that making full-fledged, truly useful docs is a lot of work, but you'd also be surprised how much you can get out of relatively little work.
Do you have a set of questions you ask yourself when writing, to check it will be useful and usable by users?
At the end of the day, it comes down to identifying what kind of audience is reading.
The ideal (and hard) way to go about docs is to be redundant/verbose/overcommunicative to target different audiences with different forms of writing.
The paretto approach is to laser focus on the one type of docs that help the most. My super dumb heuristic for this is just looking at question frequency in chat and offering the FAQ that would've answered the question.
What I sometimes try to do is link from an FAQ to a more elaborate section if there's rationales to be explained. Then the skimmer can quickly copy-paste, and the digger can go optionally read more in-depth content.
Our docs are also loosely structured by order-of-learning (e.g. "what is X", "here's pages about the most important components of X" and how they typically tie together, each page with their own FAQs, and then finally an API reference)
For mithril.js[1], I have intro ("tell me what this is and why should I care"), tutorial ("going from zero to hero"), guides ("how to setup specific things"), API reference ("what methods exist"). There's also a glossary ("what does the lingo mean") and a cheatsheet ("just let me copy and paste").
https://jira.atlassian.com/browse/CONFSERVER-14910
I will almost always shove a bunch of distinct keywords at the bottom of my docs so future searchers can find useful information. This finding information through a log(n) tree search is total bupkis. Relying on others to organize information in the same mental model as you is totally unrealistic.
The OP actually tells that story exactly. They love writing answers in slack precisely because they are ephemeral and targetted at a specific immediate need (which also makes it a lot easier than writing general-purpose documentation btw). But the persistent documentation "just decayed and eventually nobody used it." Exactly.
It's possible the right tools can make it easier to maintain (and thereby less likely to go out of date), but I don't think just simply taking a graph approach is sufficient or a very significant part of it. I'm not sure what that would be -- one obvious answer is tools that generate docs from code or otherwise tie it to code in such a way that when the code changes the docs are forced to change. But while that makes sense, every attempt at this I have seen results in disastrously unfriendly documentation (see eg "relish").
Other approaches are just to capture all that "answer the ephemeral question" documentation that is happening anyway in a way that makes it persistent and more easily searchable. A sort of "knowledge base" approach. (which, true, isn't tree-based). Some of the answers will be out of date when you find them later, but that is expected which somehow makes it less disruptive, and it's still better than nothing.
At $WORK I've got our repos hooked up to Slab, where the wiki style docs also go, so any search in Slab finds the markdown files as well.
And WRT slack, many tools including Slab let you use /slash commands to find answers. So in response to a question, you can do a /slab search, and maybe even /slab create to start a stub for a topic.
I'm guessing, it is https://slab.com/ ("The knowledge base that democratizes knowledge"), which I discovered by searching for "Slab for Slack".
It is also easier to phase out certain documentation, but writing newer, more correct documentation and deprecating the old. Of course this is still possible with a tree-based solution, but there is more flexibility in a graph-based approach.
The problem I see here is that writing docs when writing code is a different kind of mental activity, and requires some amount of context switching which developers by-and-large are loathe to do.
At least for NodeBB's API[1], we wanted documentation that kept up with changes, so we put in the hours to get a handwritten OpenAPI v3 spec written, and added a rigorous test suite[2] that not only checked syntax, but ensured that all routes defined actually returned a response with the expected schema.
We extended this later on to do the reverse; to check the mounted routes and ensure it is defined in the spec.
It's a pain in the butt when your feature gets pushed and immediately fails API tests, but the tradeoff of having guaranteed up-to-date docs is well worth it.
[1] https://docs.nodebb.org/api/ [2] https://github.com/NodeBB/NodeBB/blob/master/test/api.js
The reason is that every time you provide an ad-hoc explanation of how something works, you have a concrete context: a person, what they're trying to accomplish, and their understanding. You also have the opportunity to correct course as the person provides immediate feedback in the form of questions and comments. You have a high probability of eventually navigating to positive feedback and a thank-you at the end.
Writing documentation for an abstract audience is much, much harder, and as a result, you will likely hear a lot of negative feedback.
You think you've documented something really thoroughly, and then you hear that it took a recent hire two days to get through it instead of two hours. You start hearing, "This production troubleshooting task was supposed to be documented, but the new person said they had to figure most of it out for themself." WTF? What is "most of it?" You wrote down everything. So you talk to the new person and find out that even though they're a mid-level engineer, they only know Java and have never heard of jq, so for the part that requires pulling some data out of a complicated API response and feeding it to a different API, they wrote a 1000-line Java program complete with tests. You also find out that nobody told them how to make read-only calls to production APIs for troubleshooting purposes (you probably should have linked that info from your documentation) and oh my god in their quest to do this they somehow got their hands on the production API key used by one of the services. Equally horrifying and impressive. So that's what they've been doing for the last two days, because your documentation didn't anticipate the questions they had.
All of that would have become apparent in five minutes on Slack, and it could have been cleared up with a couple of links and examples, but you didn't anticipate any of it when you wrote the documentation. And that's because documentation is hard.
I tend to see docs and interactive QA as complimentary. If you can answer questions by linking to documentation it saves a lot of time.
I highly recommend Daniele Procida's talk about this, [0] and the accompanying web page [1]. He gives a useful taxonomy of the 4 kinds of documentation. Through the power of Unicode:
┌──────────────────────────┬─────────────────────────┐ /\
│ Tutorials │ How-to Guide │ || Practical
│ (Learning-oriented) │ (Problem-oriented) │ ||
├──────────────────────────┼─────────────────────────┤
│ Discussions │ Reference │ ||
│ (Understanding-oriented) │ (Information-oriented) │ || Theoretical
└──────────────────────────┴─────────────────────────┘ \/
<=== Useful for study Useful for coding ===>
[1] https://documentation.divio.com/
edit Didn't spot the link to [1] at the bottom of the article :P
The bane of my existence in the last 24 hours is a typical example: https://www.freepascal.org/docs-html/current/rtl/fgl/tfpgmap...
There are links that go places, but nothing anywhere actually shows working code examples, or even non-working fragments.
One of the reasons actual civilians were able to write their own programs back in the 1980s was that Turbo Pascal, and Visual Basic had decent manuals, and there were plenty of "dummies" guides as supplements.
Documentation should include working example code for each and every library, function and property supplied.
/End old man rant
Wiki linking, in that case, is just an alternative way to duplicate the content of a page on the tree. Other methods include redirects and using software that can symlink individual blocks of text to keep them in sync.
As to why documentation is bad, it's because people close to a project are not in a good position to communicate about it. Absent a large, self-documenting community, the best solution is to either have documentation be done by an employee who is a power user or customer of the system, who writes well, or by taking developers off production duty to do nothing but provide support to users and structure them into general answers and explanations.
I've usually seen those called transclusions. Occasionally someone reinvents the idea and comes up with some other neologism, often based on the word 'quote' (eg. superquote, hyperquote, etc.).
Is it really too uncool to say "hypertext" these days? That's all I'm understanding here. Write docs in hypertext. Otherwise I agree.
Wiki just on itself, where you need to browse inside to get where you need won't work. Now you need to find the right node in the graph that connects to your information.
I feel the same way about Stack Overflow. I rarely search for the answer to my question inside SO, instead google takes me to the relevant page.
So what companies seem to need is documentation in any format, hierarchy or graph, plus a really, really good search engine over it.
We're using the Atlassian wiki, for which I find search to be very bad so we kind of _have_ to work with links.
So once you accept that a graph structure is useless without powerful search behind it, the advantages of the tree approach are more obvious: it provides a basic search mechanism through hierarchy, and still allows for a graph approach (via linking and duplication of content in multiple categories).
Also, maintaining documentation sucks. So there's that...
> When is the last time you just jumped onto the Wikipedia homepage and found your way to the information you wanted?
Literally every day. If you haven't done this, the only explanation is that you haven't tried. Wikipedia's built-in search is fine (and also completely unnecessary more often than not, since you can guess most page titles and their URLs). In fact, I've had my browser's default search engine set to Wikipedia for probably 10 years now. If I want to Google something, it's either because I've already hit the Wikipedia article and it was wanting, or it's because I'm looking for something different in nature.
The claim that Wikipedia is junk without Google is itself junk.
Documentation sucks because it's hard/impossible to do well. Every change to the code can trigger cascading changes in documentation and there's no compiler to tell you which things need to change.
If we can't even keep the README up to date, when that's in the same repo as the change, how on earth are we going to be disciplined enough to go update the docs.
And the occurrence of the word "disciplined" hints at the real reason: it's a lot less fun to write docs than write code, so we don't unless we're forced to by circumstances or by management.
That's not really what we're talking about here though. Like you said, it's the concepts and informal stuff that's the problem.
I've found autogenerated stuff to only be marginally more helpful than just reading the code in the first place.
I’ve seen the “no documentation = job security” anti pattern for decades. But recently I’ve seen the “if I write something down I can be questioned better by more people so I can lay low by never describing what I’m doing how how you can reuse my libraries and methods.”
It’s a weird situation where people get promoted for “doing stuff” and not for “others doing stuff with my stuff.”
I’ve never seen any incentive for that, apart from colleagues’ thanks, which is not especially converted into promotion.
I've invented a documentation style I call "Notes for Nerds" It consists of the specific notes you make for yourself if you need to re-implement something complicated you've done.
It's example driven, very specific and includes only such verbiage as necessary to contextualize the specific technical commands or code.
I invented it because I found at Google that the best documentation was always the rough first pass stuff before it got prettied up. I specifically searched for deprecated documentation because it covered the technical exploration of the project before the final gloss was applied. Compare this to Google's public documentation if you want to throw up in your mouth a little bit.
Notes for Nerds style works smashingly. I hereby grant the reader the right to re-implement my idea anywhere you like.
Also, don't use Confluence. It is everything bad and wrong about a documentation tool. If Notion had a ticket system they would kill Atlassian overnight.
What I try to help support on my projects are higher-level block diagrams which show the relationships between major components. Hopefully that doesn't change too quickly! The lower level API should be generated straight from the code itself with a tool like swagger or doxygen if possible.
This should be reflected in the documentation.
Just few of the missing aspects:
1. Narrative style. You can't do ad hoc whatever wherever and call it a day. Most people don't have it and many are quite illiterate IMO. You need to practice this and most engineers don't like that. Hell, even most seniors don't like writing tickets IME which take almost the same time as putting garbage on Slack. I created templates on both GH and GL and almost nobody uses them even tho you don't need to think about anything but follow few rules.
2. Its quite hard to know what level of detail to put in documentation. You need a lot of experience for this - put to much, and it gets quickly outdated, put too little, and it doesn't convey much. Good documentation exists on multiple levels - as bunch of markup files "on the spot", as formal hi and low level documentation and also those are usually affecting different target groups so you actually need to design docs. Cross references are absolutely essential, the docs need to be 100% hyperlinked and in static site format (no big Word or PDF docs really, that should exist only as an export variant). But you also need a tree, because you can just say to people "here is a bunch of docs" just surf around, and good luck". So both tree and graph are required at the same time.
3. Documentation is a service. It has source code, build procedure, automatic link checking, versions, exports to bunch of formats, crosslinks, variables, macros, configuration for different environments, abbreviations, definitions, automation, includes and what not... Its quite hard to get it right. After years of struggle on different projects I finally created my own stuff [1] by bundling many different tools and configs in docker container that I use on all projects, for docs spanning 50-500 pages. I maintain that for years now, constantly (so yeah, its a job).
I am always interested in this as I write tones of docs as a part of system design role. If you have some good ideas to offer, don't hesitate.
This article reminds me of the challenges of an object-oriented decomposition of a system. The solution being something like adding aspects (see AOP), relationship links with metadata, and, sure, adding a graph structure.
However, I would suggest that reducing the space between a documentation system and the code base it is documenting is one of the deeply important missing pieces. What belongs on a distilled documentation page versus in the code is the spectrum. We have data formats listed in our documentation but those should be links to the schemas (and maybe examples) in our code base.
I'm not sure why you have that impression, both of the exemplars are user-facing.
StackOverflow is arguably more ad-hoc than a pure Wiki approach. I can already add links inside Google Docs. So the graph isn't the solution that solves this.
1. I need an entry point, if things are organized in a graph, then I have no idea where to start. Wikipedia has a landing page. For causal browsing, when I don't have a specific interested topic, I think it's fine. But If I'm new to a code base, I will prefer a start point and a step by step way to understand it. In that case, I think the graph structure fails to serve that purpose. You could create a landing page with a content table, but then that would be the same as the tree model.
2. I won't know the big picture, or the overall scope of a document.
For example, when selecting a third party library, I probably won't think of any caveats that would affect my use case. If I see a caveat section in the content table, I would pay attention to it.
My current approach is just to read source only for docs so whatever is in the readme.md and the source files is good enough if a project doesn’t have someone smart who’s into documentation.
And then like “meta docs” generated from markdown in git that links to logs of projects.
Most of the stuff I work on is internal only and even paid people seem to have bad docs as well.
Source + markdown is good enough and seems to attract more internal edits so at least material is more likely to be correct.
Any other scheme seems to fail based on the size of the group so I don’t bother trying to make a specific structure other than “use your judgement, make code readable, make sure any design work not in code is in markdown in the same repo.”
There is a need for an hybrid, as (1) is required for audit purposes, it has to be in that structure. Copy (3) has to be in (1) folder somewhere as its related to that.
(2) at the moment is just theory but not like a living document.
There needs to be a hybrid structure of all the above to make a living documentation as it sometimes it's hard to know why a change was made without knowing the project name. Can be found but it takes time and effort.
People do without internal docs for the same reason they ship without tests, Pareto principle. And writing is hard work, especially if it’s meant for other people to read.
Cheap, good, and done. Pick 2.
It’s people who don’t keep expectations realistic that wind up going crazy in this industry.
If you want your company to keep nice internal docs, you can’t be cheap. You’ll have to extend deadlines. You have to tell people: I’ll have to add another week for docs and such. Usually, it’s not worth to even bring it up. Why rock the boat?
There’s also the question of how much value they bring, usually not much.
That seems like a super useful categorization, and I intend to use it to inform my own efforts going forward - the first thing I noticed when I looked at this typology was that I generally try to write all of these, and probably the primary barrier to utility in the documentation that I write is that I haven't previously had a good sense of how to separate them out.
And Wikipedia has a tree-ish structure too, after all - look at portals, categories etc
No. The issue is that many companies still haven’t realized that you need to dedicate people to this job, that is, technical writers, documentarians, information architects, etc. Blaming the tools is like blaming a car without a driver for not moving.
To "people over process" add a new mantra to your agile way of work: "people over tools".
Tailwind is a good example of this at https://tailwindcss.com/docs.
The table of contents is worded pretty well but once you get past ground zero of using a tool being able to search for what you want becomes a quick way to find what you're looking for. Especially since the search is powered by Algolia, it's pretty good with full text search.
I am hoping to fix this by introducing Optic [0] to automatically handle generating API diffs.
What am I missing here?
Obvious stuff is hard to set up, e.g. private wiki with SSO.
It’s still easy to solicit edits, but the issue/question flow is helpful, and have some review process seems to help. Lots of people struggle with git mental models, even using the built in browser IDE in GitLab and GitHub. But most engineers seem to be able to handle.
The problem is that wikis can’t be globally editable without license events so in order to edit the wiki you need permissions on the project.
The PR flow is for the non-wiki where there’s just files in a repo that only active project users can edit but anyone can fork and PR.
So it’s a lot easier administratively than trying to grant everyone edit permissions on a repo just to edit a wiki.
If there was a decent wiki product I would love to run it and just have every staff member with edit permissions. But that’s the dilemma.
What does this mean?
> If there was a decent wiki product
Like Mediawiki.
Hmm. Have you tried any of the Mediawiki enterprise distributions?
It's knowledge transfer.
I haven't seen any bigger problem until now and I haven't seen anything that comes close to a solution. Even though, I'm pretty sure of what is not a solution and that is (mid-level) documentation.
Before I go taking that back a notch, let me clarify what I mean with that, as I've seen the cycle numerous times and have been at several of those points myself.
"This is a mess, I don't understand anything... this isn't logical. We need to document everything!
-> Sorry for that bug, I read your docs, but they were outdated.
-> Sorry for that bug, I didn't even bother to read the docs, because most seemed outdated.
-> We'll need to make sure the docs stay current, let's hire a technical writer / wiki gardener / wtf and introduce a process in our "Definition of Done (tm)" so it's all up to date
-> (Actual Development mostly halted by now)
-> Hey, welcome! Oh by the way, before your first commit, please read these 700 pages so we're sure you know what you're doing.
Talented Junior Dev: !!! — Here's my resignation."
The clear exceptions to this casual observation of mine are implicit and high-level documentation.
Implicit includes:
- self-documenting types and code ("Ronseal Code"): https://web.archive.org/web/20170426021541/https://peternixe...
- self-documenting gRPC and Swagger API schema
- self-documenting constraints
- Architecture and Schema diagrams
- Specs and Design Docs
- READMEs
There's only a very limited case where mid-level docs make sense:
- lots of consumers (public & commercial code)
- stable systems with high fluctuation operations
Other than that, this kind of documentation rots far too fast to have much net benefit at a high feature pace.
The only tools that I've consistently seen work so far for knowledge transfer are simply hiring good and communicative people with a learning mindset, keeping attrition low by keeping them happy and productive, intro sessions, code reviews and growing reasonably slow while leveraging junior/senior (as in tenure) pairings for new projects.
No kind of documentation in the world replaces the mental models of the people who architected a piece of software.
Old guy rant end.
Better to have specialized templates and page types to expose the right amount of context (eg. Wikipedia category pages, navboxes, sidebars, etc.).