Ask HN: How to write documentation for a medium size engineering org?

If you have 250 engineers (each with a non-trivial salary and total comp) then you can afford hiring or contracting a few good technical writers. That's the solution--there's no slick technology, no methodology, etc. that will make hundreds of already busy and potentially overworked engineers churn out quality technical documentation. Make a case for the value of technical writers, take it to management, build and lead a team to do it...

If you need some ammo, devex ('developer experience') is kind of a hot thing in the agile world right now. Martin Fowler has some good recent writings about how improvements to developer experience (of which technical docs are a huge part) vastly improve the efficiency and output of entire dev teams: https://martinfowler.com/articles/developer-effectiveness.ht...

Spotify apparently has invested a ton of money and resources internally into building a first-class devex platform for their teams: https://backstage.io/ Watch some recent talks about it from Spotify folks for great feedback on how it helped their teams. What they've built is exactly what you describe--self-service, low cognitive overhead to start using and sharing tools across the team.

Backstage is such a delight

We started by building a culture of documentation within our team. A good place to start is with ADRs (architecture decision records).

We also identified barriers for people to write docs on random topics. People thought writing docs would be onerous, so we explicitly encouraged low effort one liner how-tos.

Getting stuff into a wiki is more important than architecting it properly. You can reorganize it later.

You also need to refer people to the wiki to answer their questions.

Not saying we have it sorted, but for our corner of this 400 eng org is making progress on useful docs

Writing docs is also important for working asynchronously so people were happy when writing and commenting on docs meant meetings started disappearing from their calendars and any remaining calls were more focused

We are building a tool (Katara: https://katara.io) to help with this exact problem. It gives you a structured way to catalog everything in your organisation while enforcing consistency and ownership.

Confluence is a good tool for writing documentation but it gives you no way to enforce that every page of a certain type contains the same information, and it is super easy for a page to get orphaned/outdated without anyone knowing.

Katara sits on top of all of your tools and helps you manage it all.

You start by using Katara to model your org structure, i.e what teams do you have, who belongs to which, and how are they connected. You then create layouts that let you specify for a given type of page what information you expect and in what form, e.g. All product pages must link their API docs, their Jira board and their Figma project, ect. Every team page must contain getting started guides, a link to their support Slack channel, a mission statement, ect.

Then you create pages which are instances of those layouts, e.g a devops team page, product pages, feature pages, ect. Every page must be owned by a team (not an individual) which is backed by your org structure. If a team is disbanded or there is a reorg it’s not a problem, the pages are automatically reassigned (or you can reassign them manually). What’s really cool is that if a page is missing information it will create a task for the owning team to fill it in. You can even retroactively go and update layouts to require that every page of a certain type now contains new information, and all the teams that own a page of that type will be notified.

We essentially want to be the source of truth for your organisation. You still use Confluence to write/store docs, Figma to make design files, Jira for tickets, ect. We just sit on top and help catalog it all. There is no silver bullet to knowledge management but we make it easy to enforce knowledge sharing best practices in your org without a huge coordinated never ending manual effort.

Keen to chat if you are interested, we are looking for pilot customers at the moment :)

This really looks interesting! We are currently looking for a solid knowledge management solution, I might get in touch...

Would be great to chat! You can reach me at francesco@katara.io :)

I totally feel you. When I joined my current department it was pretty much the same. The engineering team transitioned from Jira to another Jira to a company specific PLM solution, and all formatting of existing docs in the process.

When I was on boarding I had to talk to so many people to get a brief understanding on all processes, tools, etl, deployment methods, APIs and so on. I made some notes on my way and felt the need to share them with the others because everyone was struggling with that topic. The only one source of truth was a 80 page pdf file maintained by one architect.

The solution I went for was markdown docs in Gitlab plus pretty HTML pages generated by mkdocs. The mkdocs material theme is really nice. This has been so successful that we have over 3000 commits after 18 months and over 1000 merge requests to the docs. The main benefits are:

* easy to use, markdown is easier that rst. Especially for non dev people. * wysiwyg editor in the gitlab webide. Most changes are made in gitlab itself. From change to merge request in 4 clicks without the need to understand git. * it is just text. In the beginning many people said that they wont contribute anything because the next tool migration will loose everything. But now the docs are in markdown and easily portable. * gitlab ci automatically lints the changes on every commit so that mkdocs doesn't break because of malformed syntax and broken links. You can only merge to master when the pipeline is green. * possibility to peer review. We have regulatory requirements for 4 eye principle of some parts, with merge requests we can easily approve and discuss merge requests.

The biggest problem is in my opinion to get our very non technical management onboard. They basically ignore to comment on merge requests or discuss changes in gitlab because it's a developers tool. Therefore they stick to stupid mailing lists and are left out in the process. But everyone from dev to ops, tst, sec is really happy.

It would be nice to have a technical writer maintaining and cleaning up the docs, but there is no budget for that, so it's will probably stay my side job.

Nothing works. Oh we have tried and tried. And there is a lot of old, out of date information scattered around. Often IT has changed the keys so nobody could update it anymore even if we knew where it still was. Then someone stumbles across it and makes a bad decisions.

im still a young engineer but few thoughts:

* consider an alternative to confluence. that monster is unreadable especially if you have larger docs. also not enjoyable to edit and pretty much leaves all docs stale and unwanted.

* Do you have a proper README.md in every code base? That seems to be a pretty standard thing to do and as a developer, something I trust the most.

In my current team, I've had a tendency to proper document things because of our team's culture and way of doing things. It's just a shared mentality of doing things right.

Full disclosure: I’m one of the founders of a product that lets you create a wiki on google drive (https://kbee.app).

We’ve been live for over 3 months and have heard first-hand about how hard it is to get everyone in your org to contribute to your wiki.

In our experience/research, what we learned was that most folks within an organization are either:

a) too intimidated by the tools they need to learn/master to contribute to their company wiki

or

b) too limited with their bandwidth to actually go through the process of updating your wiki using the tools you’ve provided.

This is usually what leads to the situations you’re talking about.

Here’s what we’ve found works in terms of creating a culture of documentation where everyone feels empowered to contribute to your wiki:

1. Use a wiki product that both the technical and non-technical employees feel comfortable using. This is super important. If the least-technically savvy teammate in your organization can not successfully use and contribute to your wiki, they won’t use it. This influence will spread to other team members as well and then your wiki will quickly became stale and irrelevant

2. Set clear expectations around how you expect documentation to be formatted and presented in your wiki. This may be something a senior leader has to help enforce/kickoff. Give folks examples and scaffolding to use to create their own articles. this introduces a level of standardization and sets expectations for what is acceptable in the wiki. It also enforces a cultural expectation of how your organization treats documentation

3. Make referring to the documentation first a company-wide initiative. A great example of this is (cliché but true) how Amazon asks for one-pagers on a topic before the start of any meetings.

Let me know if you have any feedback or questions. We love sharing what we've learned.

In a 250 org, I would propose to document the APIs automatically through swagger[^1] and use something like sphinx[^2] to get started for everything else. That said documentation is extra burden that ppl usually don't get credit for. Keep in mind that apart from creating the docs, the team needs to keep them up-to-date and that's a challenge as well. Adding "review documentation" in the definition of done could help.

[^1]: https://swagger.io/docs/

[^2]: https://www.sphinx-doc.org/en/master/

I just have to voice an opinion that swagger is not an API documentation tool. It might be a helpful tool, but it does not produce an API specification.

An API specification:

- must be diff'able with any previous version

- must have a version number

- must specify every parameter, it's range, whether it's mandatory or not, list all possible values for enumerated types, etc.

So if you use swagger, you won't be able to:

- review changes needed in your part of the system before the code is published (even in a development environment)

- analyze impact of the changes

- understand the dependencies

OpenAPI (not Swagger; that's the old name for the spec and the current name for the toolset) has all of the above functionalities. One important component is, to get the best benefit, it should be used for specification and not (just) documentation; in other words, you define the contract first and then base the various implementations (both server and client side) on it.

Unfortunately there are not many tools that support that approach; I've tried to rectify that - for Python - with Pyotr [0].

[0] https://pyotr.readthedocs.io

I'm realizing that I need to specify: I'm not primarily interested in documenting APIs, more things like:

* Preferred way of doing X

* X domain uses these tables and works like this

* This is what X feature does and here's what it looks like

Most of us work in the single large codebase, and we don't have a good separation of responsibility between the pieces yet (it's a work in progress) so for now I mostly want to document what stuff exists rather than formal API documentation.

1. Store your documentation in github wikis, not in Confluence (Confluence is slow to load. It's WYSIWYG editor is somehow as good as a simple markdown editor. It's hard to find things. It becomes a big dumping ground. Write-only documentation. By using github wikis, the documentation lives where the code lives.)

2. Don't put documentation in git. Just don't. No one will contribute to it. It'll require code review to check in a typo fix.

3. Let anyone contribute.

4. Encourage junior developers to fix bad documentation. Senior devs won't do it, they think it's beneath them. Junior devs might think it's above them. Empower them to do it.

Pick up a copy of Organising Knowledge: Taxonomies, Knowledge and Organisational Effectiveness by Patrick Lambe

Confluence can quickly get into a giant ball of mess. You need to add structure through process and create ontologies.

This same exact thought has been bugging me for a long time. Over our startup (~30ish people), we've experimented on writing on confluence, didn't work. We've tried setting up the knowledge repo [0], it didn't scale much (probably because of our infrastructure). Currently we're dumping everything to tara.ai.

What I have found is, it just hasn't worked so far in terms of discoverability and search. So far, GitLab wikis have been seamless. Archival is fine with them, but search and knowledge re-assessment has been not so easy. My ideal solution (of course, haven't done so far in our company) is to have Knowledge management system similar to Roam/Obsidian/Logseq/Athens/TiddlyWiki. But we haven't figured out any mechanisms to host Obsidian. I've been personally putting the knowledge to Roam/Obsidian graphs. Even if we figured out to host, probably team collaboration would be pain in the ass.

So far, we're dumping everything to tara.ai and GitLab wikis, hoping that someday we might be able to index these data and then combine everything for search.

This also has been a topic with few of my friends for a long time now. Knowledge management at startup/corporate level is fragmented it seems. If we had tools to self-host networked-thoughts, it'd be great.

[0] - https://github.com/airbnb/knowledge-repo

I would suggest asking engineers to document information they were seeking, when they receive it, in Confluence(under a reserved space). In that way most commonly requested information gets stored and engineers get into a habit of adding it in, in case the information is not present. Not all 250 engineers might participate at first, but when people start noticing the usefulness of this knowledge trove, they might chip in.

You would need to have a few volunteers to act as maintainers who should check regularly for the quality of the information that is being added, organize into relevant subtrees and create/update guidelines for adding new information.

This would also aid engineers who contribute improve their documentation skills as writing code is easier than writing about it IMHO.

1. Start by making it easier to learn whom to talk to.

This requires that you accept that the very thing you are frustrated by is in fact fundamental to the value delivered. The purpose of documentation is to aid conversations.

2. Make life easier for the people whom most people have to go talk to.

3. By default, documentation goes stale over time, so keep the number of docs that are expected to be up-to-date to a minimum.

Make it so that most docs have a clear timestamp and context so that people don't have to wonder if it is out-of-date or not. One example of this is https://adr.github.io/

We support the entire process of digital transfomration of topics related to product communication. Digital and analog availability of information with smart processes in the back.

We have been on the market for 35 years and are networked across all industries. Let me know if we can help you :)

https://www.fischer-information.com/en/

Until doc does not count as code, what do we expect. Not mentioning how most companies treat their code even. Haha.

Nice to see people who are interested.

1. First of all, document the APIs in a controlled fashion (versions, reviews, owner, list of consumers).

2. Allow for free form notes, but with a powerful and relevant search. I don't think such a product exists, and Wiki/Confluence is a very poor choice, the search is sub-standard, in my opinion.

3. List the owners of each subsystem (and keep this list up-to-date).

Hey,

We’re working on a really new approach to this problem (I worked at 7 different companies and ramp-up/oncall sucked every time). Our alpha users really liked the approach we’re taking.

I’d love to give you a demo of what we’re building. Could you shoot me an email (my email’s in my profile)?

How long have you been with this org? If you're new, and not familiar with being new, this is just how it feels.

I've been with this company for years, it's way more than just being new unfortunately.