Hacker News Reader

Comment by canucklady | Hacker News Reader

My current employer was sold to me as a "high documentation" place. What it means in practice is that if you're trying to do something there are 5 outdated documents describing the decision making process for how the project was run, and no documents about how to actually use the resulting software. Occasionally if you ask how to actually do a task in Slack someone will yell at you that you should have searched for a specific, obscurely named document in Google Drive, Confluence, or Github. We've tried a bunch of search tools which successfully surface the million product documents, design documents, PM reports, planning docs, retro docs and standup and oncall notes related to any feature, none of which are up to date.

View on HN

I would suggest introducing two things.

First, introduce The Diataxis framework ( https://diataxis.fr/ ) for documentation. It makes people think about documentation in a more structured way, and allows you to be more specific in the types of missing documentation. (High documentation cultures are often good with explanation but not tutorials, for example.)

Second, I would introduct the idea of a Documentation Portfolio. I have a review of Agile Documentation at https://www.ebiester.com/documentation/2020/06/02/agile-docu... and it speaks to another structure for how to build the documentation in a more reliable form and thinking more carefully about your audience for a particular type of documentation.

Diataxis looks interesting thanks for the link

Wow yeah, it puts into much better words than what I've been trying to get software engineers to do for a decade or more. Really awesome resource, thanks again kind parent :)

The nice thing about this as well is that, unlike a technical framework, you can start implementing many of the ideas of this framework without any sign on from the rest of your group. And if it works, what will eventually happen is people will say "wow, capableweb rights such fantastic documentation, we should go to them and ask for their advice on how we can all write documentation that good"

Diataxis looks fantastic. That chart on the home page is absolute gold.

Thanks so much for the link, I wish I'd had that chart ten years ago!

I'm sure these are great technological answers but this problem can be solved simply and quickly by a human.

Not every issue needs to be solved by a butter robot.

Why not employ a technical writer/documenter/whatever job title you like, even as a temp, whose sole job is to sort out the mess of documentation you have and then to write new documentation as you move forward?

My experience is that for internal documentation the time spent explaining things to a technical writer is bigger than the time spent writing the documentation

This isn’t the case for external documentation, that has to be more polished, needs sign offs and images and demos and stuff - tech writers can come in useful here

Most developers just don't like writing docs, are too busy to write them, and aren't very good writers anyway.

So even on those rare occasions when they do actually document something, the documentation tends to be pretty bare-bones and not very readable.

Good technical writers are worth their weight in gold.

Having worked with software engineers for the last 10years I can confirm, most don’t like documentation. At least 80% dont

Which is a funny statement as almost all devs like to have documentation.

If I'm simply consuming a library, I find most real-world documentation to be pretty superfluous. A full working example is usually enough to understand how things fit together.

If I'm working on a library, design docs and commented code is nice though.

It may take more time to explain it to a tech writer than it would for you to write it, but in most cases the final docs written by a tech writer will be much better.

The tech writer's output will be a Force Multiplier.

> Why not employ a technical writer/documenter/whatever job title you like

Primarily because it's a far, far more complicated job than that and you can't really hire someone off the street to do it effectively. Typically in a tech company a tech writer is going to know almost as much or more (after years of experience diving into every detail) about a given technology or application or API, and so that begs the question why not make twice as much working as a software developer and not have to sort out these types of messes?

Also job security. Anyone doing this work full-time is the first on the chopping block, and developers who are working on documentation tend to be perceived as lower status since they aren't delivering features.

What you probably really want is a librarian (with some degree of technical background). We at least had one for a while for our sales/marketing docs--which are separate from customer-facing technical docs.

> you can't really hire someone off the street to do it effectively.

I disagree, I've been apart of a few companies that have done exactly this. I would suggest not presenting this argument as an absolutism.

Summary: HIGH Documentation = HIGH staleness + HIGH loss. HIGH staleness is because nobody wants to do it (status is lower). Also… nobody else can do it (full understanding of what is being documented is needed)

So, to solve the first staleness part, there is only two ways: raise the documenter status, or make it somewhat possible (easier?) for someone else to do at least a part of it. are they both really that hopeless?

PS: to solve the second loss/discovery part, I think we are heading for that AI powered simple "unified search" experience.

AI can't solve search. If you look at how google did it, they bullied and cajoled site owners to add detailed metadata to the top of pages. It's not magic, it's creating incentives for people to create documentation.

First, I’m assuming the documentation is already updated (i.e., 1st part is OK = no staleness)

Second, the whole point of AI NLP search (i.e., 2nd part = loss) is that it does not need metadata (which was the basis of the now mostly abandoned semantic web approach to KM).

The better question to ask is if AI can do the ghost writing that high-documentation culture needs. This is an interesting hypothesis for GPT-4

'bullied and cajoled' is an interesting set of verbs to use here. Is there a reason not to use metadata? Doesn't it make the web easier to index, and therefore easier for everyone to use?

The age old issue. There's oodles of data at how people don't keep doing it because it's toil to them.

A human has to write this type of metadata.

If you don't include the right metadata, google won't rank you highly. If you include the right metadata, your content will get higher rankings and the snazzy preview cards on different social media platforms.

Metadata is good! There are structural incentives to be mediocre though.

>Is there a reason not to use metadata? Doesn't it make the web easier to index, and therefore easier for everyone to use?

Yes, there's a very good reason not to use metadata: it's extra work, and it's not very fun, just like writing docs for software. So people don't want to do it because it isn't "sexy" (and there aren't very good incentives to overcome people's reluctance to do that work).

Because of this, just like any job that people don't really want to do, you have to "bully and cajole" them into doing it.

You need to

1) Link to the documentation in your tools

2) Ensure everyone has edit access to the linked documents

A feature of docs-as-code is having the (typ. markdown) docs live in the code directories. Leaving no excuses for tech people not to update them, or at least insert a TODO.

Another idea might be, whenever a new feature is closed out, auto-allocate some percentage of its implementation time to documentation, and schedule an interview with a tech writer.

You explained why no one wants to take the job in the typical company. They would be disrespected, and likely soon fired.

But a different question is, why is no company trying to do this differently? Like, hiring one good tech writer to maintain the company documentation, and paying them as much as they pay the developers.

> But a different question is, why is no company trying to do this differently?

I once worked at a company - in a different domain - that made a conscious decision to make this kind of hire. It worked incredibly well, and I never understood why more companies didn't do it.

The context in my case was the Australian offices of a management consulting firm (BCG). The Melbourne and Sydney offices hired what were called "editors", brought on at the same grade as the consultants. Not editing as in correcting grammar. But helping the consultants improve the logic of the arguments in their slide decks: so they were logically consistent, easy to understand, and actually addressed the clients' issues. I was a junior consultant back then, and we were constantly pushed by our managers "have you seen Yvonne?" [the Melbourne editor] when preparing for major presentations.

I would love that job, I'm always going back to presentations and finding better ways they could have made the point and identifying missing context what they would need to be more relevant.

A previous team I was on ended up with this role. Strong writer with no technical skills joined the team and worked hand-in-hand with engineers fleshing out docs. It was productive for the engineers because they needed to articulate the ideas very clearly. The writer has been attached to that project now for 6-7 years at this point, and could probably stand in as a support engineer for some problems. It was a little painful getting HR to approve a tech writer getting paid close to an engineer position (this was after a few years).

I do like the sibling comment calling for a librarian. I imagine that would pay a ton of dividends if the librarian was motivated and got support.

> differently? Like, hiring one good tech writer to maintain the company documentation

He assumes that "full understanding (into every detail) of what is being documented is needed" (as I put it). So, the new hire will never get it right 100%. he will both struggle and annoy others (to forever enlighten him), which is a fair point.

But it is not black or white. Others here have more positive experiences

As a tech writer, I think this is because it's hard to concretely quantify the value that a tech writer brings, and thus it's hard to make a clear business case for.

Some companies do value technical writers and pay them as much as engineers, but they are still pretty rare.

There are three things that I think are preventing technical writing from being more widely valued:

1. Software companies tend not to distinguish between technical writers who are good at English vs. technical writers who are good at engineering, understand their audience, and can articulate complex ideas to that audience effectively.

2. Technical writers who are good at English make about half as much as technical writers with engineering skills, but they also muddy the hiring waters and drag salaries down for everyone else.

3. Most corporate-people think because they can type up a decent email they can write technical documentation themselves. They're usually wrong on both counts.

Probably not the same pay as developers but the scenario you describe is already true in most regulated industries, where some regulated body actually asks for the docs on any given product.

> Primarily because it's a far, far more complicated job than that and you can't really hire someone off the street to do it effectively.

Library science is a popular area of study but the job market isn’t great and neither is the pay. Lots of people to choose from here even without poaching from existing libraries.

> and so that begs the question why not make twice as much working as a software developer and not have to sort out these types of messes?

You partly answered your own question: perhaps you pay the librarian/documentation writer too little? ;-)

Seriously: letting the documentation to be written by such a person won't be as much a cost-reducing measure, but instead mostly an approach to improve the documentation quality.

Usually a tech writer is also some sort of PM role that gets to chase developers and get them to explain what's in their head. Sometimes you have to have such a person on board.

I once did an internship that involved chasing down senior devs and generating documentation for them (needed for FIPS certification). It was a great way to learn about the tech stack. Suffice it to say I didn't have any PM role at all.

I'm not saying a technical writer should have a PM role, but it's a similar role

In my opinion, you shouldn't ask technical writers to manage projects anymore than you would ask engineers to. Both jobs are complex enough without adding an entirely different full-time job on top of the work.

Get the brain dump when it's all still fresh in the head !

> Primarily because it's a far, far more complicated job than that and you can't really hire someone off the street to do it effectively.

This comment is absolutely true and many, of not most, companies fail to understand it. I think the problem stems from corporate-people thinking, "Why should I pay a writer when we all speak English (or whatever language) and can write it ourselves." And that's why so many companies have shitty documentation.

> ...so that begs the question why not make twice as much working as a software developer and not have to sort out these types of messes?

I was a software engineer for 30+ years and got completely burned out on it, so I left engineering to do technical writing. So far, I like it much better because I have far more control over my time. In my experience so far, the sorting-out-messes work is about the same in either field. Both jobs are pretty complex. I also make exactly the same as I did while working as an engineer.

I think the secret to not being first on the chopping block is to show you're delivering value to customers and internal teams. At least I attribute that to my survival through multiple layoffs so far.

> I was a software engineer for 30+ years and got completely burned out on it, so I left engineering to do technical writing

How did you make this transition? Any credentials/certifications you needed? Did you transition within the same company?

My college degree was in writing, so I used that and a portfolio to make the transition. During interviews, it seems my software engineering experience and portfolio are more valued than my writing degree though.

what's a butter robot? (I feel like i'm setting up a punchline for a joke somehow...)

Probably a text to speech fail (edit: errr speech to text). Bot or robot == butter robot

It’s a reference to this: https://youtu.be/X7HmltUWXgs

Haha that’s awesome. I had figured it was just a typo for “better robot.”

Since the last episode we have also a lever robot

Thanks for asking this. Was too afraid to ask haha.

Sorry, should have been clearer - butter robot from Rick and Morty. An entity who has one menial job to do and that is the reason for their existence.

In this sense it was meant that the technological solution - auto documenter tools, etc - they're the butter robots. I was making the point that rather than just shim Yet Another Tool into the stack to do a single but important job (pass the butter / write the docs) perhaps give it to a human who will a) do a much better job and b) reduce the complexity of the stack.

Apologies for the confusion.

I'm surprised how this comment thread took off. Looks like there's plenty of support for AND against. I simply meant to make the point that for some problems humans do better than machines and it can be more efficient in the long term to look away from a tech solution to a human problem.

Er, the butter robot joke was a throwaway line about the exential crisis of a robot that carries butter, not a blueprint or metaphor about how to conduct software development, please for all that is holy don't try to read more into that than is there...

I’ve advocated for hiring a librarian but I haven’t ever been able to make the case successfully. So yes, I agree with you, but the “how” of structuring and the “who” are orthogonal.

This is about prioritizing the documentation you write.

> I'm sure these are great technological answers

They're not, actually, as is obvious if you actually read the links. They're human process approaches.

> Why not employ a technical writer/documenter/whatever job title you like

Who would supply that person with the information required to write the documentation?

Subject matter experts, to the extent that the writer isn't one already.

In my opinion, companies should hire subject matter experts who can write rather than just someone with an English degree. I've fixed a lot of terrible documentation written by English majors with no engineering background.

I came here for the butter robot.

What's the difference between Diataxis and the Divio documentation framework? https://documentation.divio.com/

Indeed, they look identical, even down to the font used.

Diataxis seems to be a fork of Divio.

But according to the network graph on GitHub, Diataxis seems to be more active, although both of them still receive updates.

From the GitHub page, it looks like Divio is actually a fork of Diátaxis.

divio/diataxis-documentation-framework is forked from evildmp/diataxis-documentation-framework, and evildmp is Daniele Procida, the creator of the latter repository and the maintainer of diataxis.fr.

Thanks for the investigation!

Seems like the Diataxis author came up with the framework while working at Divio: https://github.com/evildmp/diataxis-documentation-framework/...

The diataxis seems quite interesting.

I see how it applies to documents which describe things as they are, but I'm curious how it would classify forward looking documents like technical designs, strategy and vision documents, roadmaps, and mission statements.

Whoa! I love a good 2x2, and the one on the Diataxis home page is great!

Adding a caption here for anyone on a screen-reader, before I give commentary on it:

  * X-axis: "serve our study" vs "serve our work"
  * Y-axis: "practical steps" vs "theoretical knowledge"
  * which gives 4 quadrants: how-to guides, tutorials, explanation, and reference

So I'm joining a new place recently, and it's another one of those "documentation-heavy" places where (of course) every new hire conducts the ceremonial ritual of updating the docs wherever they could use improvement. I really like having this ontology in my head now; I can see it being very useful.

Also, I wonder if the relative distributions of each one could tell you something about the team's health — or even just the kind of work the team does?

For example, between two teams who are both documentation-heavy, what does it means if one team's docbase is 80% tutorials, whereas the other team's is 80% reference guides? It would be fascinating if anyone's already given thought to relative metrics/heuristics like this.

Almost every developer I've ever seen defaults to "explanation." Reference is there to the extent that it can be auto-generated, but most companies are relatively light on reference. Tutorials are pretty rare in smaller orgs - it rarely gets prioritized.

How-tos are an interesting case. I see a lot of informal how-tos in slack. However, how-tos also are the most likely to become stale because if they're needed often enough, they tend to become automated in part or whole. It is by its very nature transitory.

This does not help with outdated docs.

> Confluence

There's your problem. The only use case for Confluence is when you want to hide information, but credibly claim that it's documented.

Yeah, I push back when people say "can you document that API you wrote in confluence?"

It is such a stupid idea that it makes me question leadership.

Some things are good to document there, but generally, if you're documenting code, you should do it in the code.

I'd take confluence over google docs because of how bad Google Docs' search is surprisingly

Do you mean searching within a document, or searching with google drive? I've found that google drive search is incredible, they've done a great job of indexing everything.

They have no content excerpts in search results, so it doesn't provide any help for finding which result is the one you want.

It also weights titles incredibly heavy, which combined with the previous part led to me not even noticing it searched the document body for years.

Interesting. Not that long ago we moved everything out of Confluence into Google Drive because GD search worked. Confluence search was horrible to find docs I knew were there.

Confluence’s search is so bad, it might as well be a Write Once, Read Never tool.

That's because no-one bothers to read the documentation linked in the search field.

Protip: Use wildcards extensively. Differently from Google, Confluence search considers keywords entered to be limited by word delimiters. so foo matches only foo, not barfoo or footer. Use them with some wildcards, foo, and the search results starts to make sense.

Oh yeah it's not great either I agree. But the use case for Google Docs is somewhat different in my mind. It's good for collaboration and discussion, rather than a source of truth to describe "how things are right now". It's annoying if you can't find a particular document immediately but it's not the end of the world. Discussion on a Google Doc will happen for a few weeks or a quarter, then die down and the doc will seldom be looked at again. You might link to it from tricky parts of your codebase but it's not essential to a high-level understanding.

Confluence and other wiki systems are clearly meant for longer-lived documentation and canonical information. You should link from or attach your working documents (spreadsheets, slide decks etc) to your wiki documentation for people to discover why certain decisions were taken. But if the wiki's discoverability is poor or it's not well-maintained or regularly reviewed, it's basically useless.

Not a fan of Google docs either, but I recently discovered CloudSearch which imo does a better job at searching Drive (and searches emails too, and few other places).

link: https://cloudsearch.google.com

Confluence (and all of the similar products) can be used successfully, but you need the teams to agree on and enforce a logical document hierarchy. It’s not really difficult to organize a company wiki into teams, projects, and other logical divisions if you make it a priority.

The primary failure mode I see is when people just throw random documents into Confluence wherever convenient at time of writing and never go back to logically organize anything. One symptom of this is when key information is being recorded in a hundred different people’s “Personal Space”

Taking even half a day to organize the average Confluence makes a huge difference.

If using the lowest friction or default path in a tool leads to bad outcomes, that's a problem with the tool. Not the user.

You create the lowest friction/default path according to your company's needs.

There isn't a universal note taking application that comes pre-organized for your team's use case. You have to put some work into any tool you use.

I disagree. The point of a tool is to reduce work.

Most teams and companies aren't special snowflakes that need individualized organizations, and document hierarchies. There can be such a thing as sensible defaults that you customize or tweak later (no idea if Confluence ships with that - I've only ever seen Confluence installations in their already-screwed-up state). At the same time, an inexperienced user staring at a fresh Confluence install isn't going to get the organization correct right off the bat.

If you have to put in work upfront before the tool is even halfway useful, it better be really damn good after that. Confluence is not.

Disclaimer: I am a consultant working for an Atlassian-centered consultancy. I do a lot of Confluence-based projects recently.

You would not believe how special some use-cases are, especially when you work with organisations that have highly regulated environments. I've seen anything from markdown files in a git repository being semiautomatically created in a Jenkins run to an organization having built essentially their own wiki software because nothing on the market fulfilled their need at the time (now 5 years later, they realise no-one uses that thing because it is just unintuitive). I have seen organisations that have no content oversight and some who had a whole department of "content czars", whose sole job it was to keep their documentation fresh and updated. I've seen organisations that had strict rules on approving each individual change, with complex approval workflows.

If you have never documented anything, Confluence may be overwhelming, but so will every tool that has "sensible defaults", because before too long, you will start hitting the envelope. Documentation software is not like a MacBook that you just buy and start using, you always need some level of customisation.

So, is Confluence damn good? No - there's a lot that could be improved. But from the mediocre solutions on the market today, it is one of the better choices.

The lowest friction is not doing any documentation

Agreed, we use Confluence and use an agreed upon structure. It works fine from what I've experienced.

Can you share your structure? Or do you know any good open-source ones?

I personally have never seen it work, but of course I've also only seen a handful of data points.

Best advice I can think of, avoid more than three indentations in the outline.

> The only use case for Confluence is when you want to hide information, but credibly claim that it's documented.

That describes Sharepoint more than Confluence.

Sharepoint is what you settle for when you can't convince your company to spring for Confluence.

Sharepoint is what you settle for when you hate the company, coworkers and yourself.

Does it matter what tool you use to write documentation? Confluence gets a lot of sh*t because its in the grown-up camp of tools but I know people who've got problems even with nano.

It's incumbent upon all users or members of the team to use the common tool along with agreed upon standards. Otherwise even if you wrote documentation in your own hemoglobin, no one would touch it either.

Some manager prob chose _________ as the tool for ticketing, documentation, etc not because it was good at ______, or _______ but because it fulfilled their action plan to have something, anything in place so that if the universe goes supernova, well some stuff was written down.

In my journey it seems that nobody is willing to criticize Edward Teach for the lousy treasure map he left, but rather we make fun of those who're still looking for his stuff.

It does matter because the issue with wikis (not just confluence) is there's no approval or review workflow. Imagine trying to write a large program in which everyone could just commit at will, with no review process whatsoever, and where nobody had made any decisions about design up front. There'd be duplication, dead code, the organization would be crazy.

That's the average wiki. It's a commons and a tragic one. To make docs work you have to treat it more like a codebase: clear ownership, standards, review processes, approvals, up front design, refactoring efforts etc.

> To make docs work you have to treat it more like a codebase: clear ownership, standards, review processes, approvals, up front design, refactoring efforts etc.

Maybe true in large orgs.

But for smaller companies what I've seen is usually paralysis.

e.g. someone notes a problem (maybe just a typo) in the doc. Can they fix it within seconds? If instead they need to raise a ticket then most likely it ain't happening. They move on, and the next person experiences the same problem.

IMO the default should indeed be towards everyone committing at will. Yes that will result in the occasional snafu. Fix that when it happens. (obviously not good practice for the operating manual for a nuclear power plant - but for a <500 person Saas company it is).

Disagree. A ticket should be created for any change, no matter how small. It takes seconds to write a title, body and hit submit. I've seen those small ad-hoc changes cause havoc because someone forgot to escape a single quote or didn't realize tabs were necessary and replaced them with spaces.

The default for Confluence is just that, everyone commits at will. There is no structure, tons of duplication, no standards when it comes to naming, formatting, audience, etc. I'm a huge fan of markdown/plain-text solutions, only because linters can be run that force you down one happy path. I don't believe Confluence has linters at all.

> A ticket should be created for any change, no matter how small. It takes seconds to write a title, body and hit submit.

A ticket represents a process (otherwise it has no added value over git commit message) and thus creates much more work than a couple of seconds.

> A ticket represents a process

Yep, and that process also involves other people, to review/ approve the fix to the typo.

It then goes from being a few seconds of elapsed time and actual time (to just commit a fix to the typo) to taking hours, days or weeks of elapsed time and hours of actual time and forcing context switching on, and interrupting the workflow of, all of people involved.

Yes, handoffs have overhead and create process waste. They are a process smell and should be investigated for ROI.

Ticket, or pull request?

Mandating a Jira ticket for simple typo fixes is overkill. But if you make it easy to create a PR directly on the documentation file, without leaving the tab, I don't see an issue. This is already a Github feature.

We did PR's on documentation files at my last place, it worked but it was more painful than getting reviews for code PR's. Tickets work because they can be reasoned about, shoved aside, brought back into the limelight, updated, other tickets can easily be related to them as more information is discovered, etc.

Overall the comments on this page fall into 2 camps, people who've tried it all and found what works is discipline and those who are still trying it all.

There are some really nice git-based wiki systems out there, and one is built into GitHub and GitLab. If you want that type of workflow for your wiki, it's easy to get.

A couple of jobs ago (so we’re talking like late 00s) I rolled an internal wiki system on top of mercurial.

It was a directory of files - I think plain text with a few wiki shortcuts, but might have been some sort of early Markdown.

The editing form was basically a text area on top of mercurial.

Similarly things like the edit log were basically dumping the mercurial output into html.

No clue how long it lasted, but it was still in regular use when I left in 2012.

Wrote the whole thing in a few afternoons.

The last thing I want is to raise the friction for writing down documentation.

It's hard enough to get technical minded people to contribute to a git (or style) based knowledge base.

Pick your poison I guess but I'm quite happy to have testers/BAs/directors/etc able to quickly jot down thoughts roughly than have it disappear into the ether.

I mean. I guess? The difference between having it written down in Confluence and disappearing into the ether is academic though. Either way nobody will ever find the information again.

> The last thing I want is to raise the friction for writing down documentation.

A better solution might be that anyone can write the documentation, and there is a maintainer who constantly refactors the wiki to keep it legible. Makes sure the information is not duplicated, adds hyperlinks to things, etc.

Internal documentation is everyone's responsibility, not a "tech writer". External should be written by a professional, agree.

"Everyone's responsibility" sounds like an euphemism for "no one really cares". When people actually care about something, they hire an expert.

Why do you hire software developers, instead of making software development everyone's responsibility? Is that because most people suck at software development? Well, most people suck at writing documentation, too.

Well there is (ftr I dislike confluence, but come on) https://marketplace.atlassian.com/apps/1216387/approvals-for...

Interesting but is it really used? Less than 1000 installs for a product as popular as Confluence?

Most use Comala: https://marketplace.atlassian.com/apps/142/comala-document-m...

Ha I hear this a lot but the alternatives I have seen used are frankly just as terrible. I much prefer Confluence to a hidden web of Google docs.

For my team we’ve switched to using GitHub markdown files with flowcharts: located within the folder containing the subsystem it describes.

Google Drive, the ultimate information black hole. Anything that gets sufficiently close will be sucked in and never seen again.

Confluence is garbage

We use Mark[1] to automatically create Confluence pages from Markdown documents in our git repos. So we can have a review process for documentation changes, the documentation of the code can be in the repo with the code, and yet it can still be accessed without having to give permissions to view the code repo! Helpful with a proprietary monorepo.

[1] https://github.com/kovetskiy/mark

So simple an idea, it makes you wonder why it's not already a core feature in Confluence.

Bitbucket wikis basically do this.

Did I miss it or does the article specify what tools Tremendous uses for their documentation?

Yeah, the search function in my company's confluence is abysmal.

There have been times I know a page exists and I even know the title/content of the page and yet still I am unable to find it via the search.

So which tool do you like?

I will say the obvious: documentation sucks because good writing is a highly skilled activity that takes a lot of energy for most people to do, AND because keeping it up to date takes a lot of time, no matter how good you are at it, AND because leaders of companies don't want to spend ANY money on tech writers.

That's it. No mystery.

(BTW, this would also be why company financial records would suck, if management decided to save money on accounting staff and have all employees just kinda do their own accounting for the company. I SAY HIRE A SCRIBE FOR EVERY TECHNICAL TEAM!)

Ironically, it is much easier for me to write good documentation if there are a couple of meetings scheduled around it. Having 3 joint sessions with at least another person is great, first to get the "bone structure" of the document, then to confirm that the "flesh sits right", and finally to resolve any unclarities that might be left.

It sounds like you're describing something very similar to pairing (to write) and peer review on code, but for docs, rather than "meetings"?

> good writing is a highly skilled activity

This. Yes, having everything written down and searchable is definitely a good goal. However, in my experience, the people in most companies have very different skills and few are good writers. So it probably takes a lot of time to create an organization that has a good process for creating great documents, let alone to transform an existing organization which can do so.

I think it's important to realize that a lot of documentation is duplication. It duplicates something expressed in code, in configs, in structure, in people's heads, or in the real world.

Duplication can be useful. But the more of it you have, the greater the maintenance burden is. (The main exception is documentation that is not supposed to be kept up to date, like a daily journal or blog posts.) So I think it behooves people to be very careful about adding documentation. Because as you say, it can turn 1 problem into n problems.

No, code actualizes the intent of the documentation and the product. The natural language description of a product shouldn't need to be discarded in lieu of some machine language.

The lingua franca of ideas is natural language.

Fair enough, but you still end up with 2 separate ways to express things. And I have yet to see a company that changes the documentation first and then derives code changes from that.

Usually tickets are written, code is changed. Updating existing documentation is an afterthought at best.

Personally I prefer any formal or semi-formal documentation (e.g. Swagger) over a Confluence page any time of the day.

Sure. But where some see the lack of updates as some sort of moral failure, I think it's usually a sign that there is a process problem. The documentation was supposed to solve some sort of problem, but the fact that people don't update it is usually a sign that either it wasn't a real problem, that documentation wasn't the right solution, or that there's a broken feedback loop in the team's process.

I built this documentation/testing framework to do just that: https://hitchdev.com/hitchstory

I realized one day that the specs, tests and how-to markdown documentation I wrote all used the same examples.

From that I derived the idea to create a "spec" DSL that could both be run as a test and generate markdown (with screenshots, etc.) to make nice high level how-tos.

Cucumber has the same sort of idea but the DSL really isn't suitable.

I think most of us approach documentation without intention. We write to express the idea, but fail to consider how the documentation fits into the deliverable.

For example, one could structure things where the English documentation is the deliverable. The code merely serves to actualize the document. In this world, we would consider the act of writing documentation of paramout importance, whereas the code is an implementation detail.

I think software as a discipline is distinctly undiscipled about these sorts of concepts.

Code is more complex than documentation specifically because it needs to deal with every issue. If you're going to cover every edge case in documentation, just write the code. Tools like OpenAPI/Swagger blur the lines.

Documentation is useful when it's job is to help understand why the code is the way it is, what problems are trying to be solved, and what constraints the devs had.

I am familiar with the notion that programmers are people who turn documents into code. I just think it's rarely true and even more rarely necessary.

Personally, I don't think documents have intent. People have intent. Sometimes they write down some words as an approximate way to communicate their intent in that moment. But if there is a conflict between the written document and the actual intent, it's the human intent that matters.

I also think that drift between written description and actual intent is good and desirable, because it usually means that the humans learned something about what actually needs to happen. So to the extent that documentation acts as leg irons for intent, I'm generally opposed to it. The faster our teams learn, the better the things we make.

Do we work for the same company? :P

Confluence has been the bane of my attempts in finding any relevant docs. Which one is the source of truth? Which one was a draft written by an overly eager to make a first impression, new employee (who is no longer with the company)? Don't even get me started on saving meeting notes to confluence.

These days, I maintain my own knowledge base on Obsidian. If there's ever any confusion or request for more information within the company, I copy-pasta the relevant note from my obsidian bank to whomever person or whichever confluence page they deem the source of truth.

Do you have any tips on how to maintain a developer's own knowledge base in Obsidian? I also use Obsidian but I currently use as more of a dumping ground.

It's actually quite simple even without using some of the advanced features: What I do is create a directory structure for each domain as I explore them. I.e.

Toplevel: - Work -- Job A: -- Daily notes -- Services -- Auth --- overview --- login flow -- Client -- Logger -- Job B: -- Daily notes -- Architecture -- node -- react -- etc

(edit: sorry about the formatting)

As the scope of your work expands, you add another sub-directory or file where necessary. Once it starts to grow in size, you can start making insightful connections via [[keyword]].

Furthermore, you can pretty much take this knowledge base with you, wherever you go, by uploading the vault file to your google drive and accessing it locally via SMB. Automatic save/backup.

So you are now the gatekeeper?

If an individual employee is going to put all that work in without being asked to or being given scheduled time to work on it they should get something in return.

In the above scenario, I do not understand what they are getting in return.

Are you saying this person should be hired/paid to store and retrieve the information being sought out?

Being as you put it the "gatekeeper" advertises their importance to everyone in the company who needs to know about the processes, making advancement easier and guarding against anyone thinking they are not necessary.

Unless they were asked for, no? You aren't a snowflake and you aren't own praise for something you weren't asked to do. Maybe you work is great and valued (gold star), but your point of being owed something is just a bald face lie to yourself.

Maybe more like the font of knowledge?

I do the same except in org mode. I’ll export to markdown as needed but generally publishing documents is a secondary goal to empowering and decreasing the burden on myself.

Design docs for each and every feature has turned out not to scale for my current team. Larger, multi team features demand consolidated documentation, but for internal changes we rely on quick meetings as code reviews. Part of me misses the ceremony of the round table discussions, but the real difficulty is keeping track of why changes happen. Documenting processes and cross cutting concerns is a must have, but keeping track of all changes across quickly moving teams… it’s no surprise so many teams are just rife with tribal knowledge.

What many dont realize is that documentation is tech debt. You can spend a lot of time and write a lot of documentation and have to also spend time updating it.

I have worked with teams that focus so much time on design docs and insist that everything has to be documented. Pace of work is slow. Documentation and designs became obsolete due to shutting down of services, change in architecture, refactors.

The best form of documentation is code.

I think it’s a bit odd to call it tech debt. I’d say something more like “documentation is part of your tech, and needs to be maintained like the rest of your tech.” It’s only tech debt if you decide to not maintain it.

Agreed. Documentation has upkeep costs as the software and environment change.

> The best form of documentation is code.

Documentation as code might be a good alternative path, using the same tools and processes as software development and embed documentation tasks into engineering workflows. More insights in https://about.gitlab.com/blog/2022/10/12/five-fast-facts-abo... how the GitLab technical writing team collaborates.

Note: GitLab team member here.

Yeah, that sounds horrible yet familiar. Regarding this part:

> Occasionally if you ask how to actually do a task in Slack someone will yell at you that you should have searched for a specific, obscurely named document in Google Drive, Confluence, or Github.

When I'm the person being asked and know of the doc in question, here's what I try to do instead: I ask where someone searched for it. Then I update that place to refer to the correct document (and do some light refresh on the doc as needed). This works whether or not they tried to look before asking. If they did, well, now the next person who does that will just find it. If they didn't, I'm making them look before getting an answer. Maybe they find it, maybe they don't, either way I'll help them in the end if they're willing to look.

You really have to be religious about enforcing a single source of truth for any information and this gets a lot better.

There are docs and then there are Docs. I think documents should be treated as source code. Go through a proper PR process so that you know what the latest and greatest is. Maintaining a wiki is one of the worst ways to document. It just creates a sprawl that is hard to control. I deal with it on a daily basis but have had little success with getting my team moving to our source control system for documents.

It's actually hilarious how Google - a company famous for it's search engine - made Docs, who's search function cannot find anything.

I live and breathe using Google Docs search functionality. It's my main way of finding files scattered across 10 years of folder hierarchies. It works great.

What do you mean it can't find anything?

What I find is that it's very "dumb", almost "... WHERE filename LIKE '%query%'" dumb. Example might be searching for "<Cool Project> Technical Spec".

- Can't find "Technical Specification" in the "<Cool Project>" folder.

- Can't find "Tech Spec" (and vice versa)

- Can't find "<CoolProject>"

Is there some "enable real search functionality" checkbox I've missed, or am I just doing it wrong?

Yeah, I think you're "doing it wrong" as much as I hate to say that, sorry.

Search is keyword-based, like large-scale search is pretty much anywhere. Expecting "specifications" to match "spec" is expecting too much, same as expecting half your search to match a folder and the other half to match a file the folder is in.

The main thing to keep in mind is that search is content-based, not just filename. So instead, search for key terms you think are in the file, as opposed to focusing on folders/filenames. Start with one or two, then modify or add as necessary to narrow down.

True, although this is Google we're talking about and Google [web] search is smart enough to do a lot of statistically-driven NLU-type things on top of pure keywords...

(I currently work at Google, but not in Search or Drive / Docs...)

Google Photos search is magical.

So basically you have to search google docs like you used to search the web before google came along and revolutionized search?

> Expecting "specifications" to match "spec" is expecting too much

When I use Google to search the whole web, it DOES match. So how come it doesn't work when I search a small document?

And it’s still miles better than Confluence.

Are you using search feature in Drive? I've been using CloudSearch, which I heavily rely upon

I work in medical devices so we have to write a lot of docs. But they all disappear in document management systems where you can't find anything if you don't already know where it is. Are there no document management systems that are actually useful?

git works for us across business/electronics/electrical/mechanical/software.

The exception is daily supply chain and accounting, which due to factors like urgency, multiple stakeholders per order, high pace of handover, external system integration, multilingual presentation requirements and nontechnical users we prefer a dedicated web based system with more of a real time focus with event hooks (eg. notification, translation, verification).

What kind of docs do you have in git? Git doesn’t work well with binary file formats like Word, Excel or Visio. You can do it but the diff won’t work.

Markdown + .webp, mostly. Some 'TeX.

Ask yourself how many document management systems are selected after rigorous tests of actual usage vs those selected after sales presentations and schmoozing. That should give you your answer.

And if that's ambiguous, then ask how often your company penalizes people for making the common but wrong choice versus the uncommon but wrong choice.

This applies to pretty much all enterprise software. It’s rarely selected by or for the benefit of the actual users. Usually it’s selected for the benefit of management (for example reporting) or for friends of management.

A good secretary (or a bunch of them) and filing cabinets.

You may still not know how to find anything, but they will.

Like a lot of other things, this has suffered from computerization making it yet another small part of everyone's job (which also increases context switching, the amount of shit you need to know and keep track of, and generally makes jobs more stressful) rather than a specialty that's the main focus of a few workers.

The benefit (get to stop paying some employees) is easy to measure, while the harm is not.

"Like a lot of other things, this has suffered from computerization making it yet another small part of everyone's job (which also increases context switching, the amount of shit you need to know and keep track of, and generally makes jobs more stressful) rather than a specialty that's the main focus of a few workers."

Nicely said. I am getting really stressed out how complex things are becoming. It's already hard to keep up with git, Jira, Bitbucket, AWS, k8s, Helm, JS frameworks, databases and whatsoever. But then add hard to use document management systems and f...ed up processes that are mainly designed for management to get nice reports and not for productivity. Now you are a bad developer and you are a bad document management person because it's simply impossible to be good at all this stuff.

I am constantly preaching to management that we need specialized tech writers and specialized devs that are good at their respective. But I guess it looks cheaper to waste time and energy of engineers on stuff they aren't good at.

Sometimes (often?) an outdated document is still great and useful and much better than no documentation at all

Equally often it’s worse and a waste of time. And you won’t know which world you’re living in for a while.

I've yet to find an outdated doc that makes the situation worse (unless you assume it's correct and up-to-date, which you should never do with anything anyway). There's a reason we like RFCs even if they only represent a decision in time.

Everyone wants accurate updated documentation.

Nobody knows how to accomplish this.

Whatever the solution is, if one exists, I'm sure it involves a lot of work keeping documentation up to date.

They know how (documentation is embedded in/generated from code) they just don’t want to.

Out of date doc is worse than no doc, because it makes you feel all warm and fuzzy right until you footgun.

> My current employer was sold to me as a "high documentation" place. What it means in practice is that if you're trying to do something there are 5 outdated documents describing the decision making process for how the project was run, and no documents about how to actually use the resulting software.

How is this not inevitable if your goal is to always write things down? It seems like the way for document to be accurate is to keep the scope small and if you want everything in scope then it's going to contain a lot of outdated information.

You basically need to deprecate and eventually probably take offline outdated docs. There's something to be said for the historical record but if it's indexed--and if it's not no one will probably find it--it's going to compete with current documentation for search.

There's no easy answer.

Large organizations in many sectors employ professional records managers for this reason (and many others). Every record has a “lifecycle” and it is deprecated and discarded after that.

So update the documents?

I mean, you're clearly not describing a high documentation culture, you're describing a culture that underinvests in intra-organizational communication.

I run a high documentation, low meeting culture by necessity (we operate in five time zones around world). Meetings vs docs is remarkably similar to the decision between paying for office space vs paying for occasional team retreats. If you run a fully remote company retreats are almost always a better use of your money than leasing office space. But you still need to pay for something.

Similarly with meetings vs. process documentation. If you're heavily remote and spread out I'd say you should cut down on meetings and being high documentation is the better choice. But again you still need to "pay" for something - you save time on meetings but you need to reinvest at least part of that time into writing documents.

Another bonus of documents is that they scale better than meetings. McDonald's doesn't deliver the same big Mac in every corner of the world by holding a lot of meetings. They have a book that goes out to all of their thousands of franchisees. When they want to add another thousand franchisees, they print more books.

If the documents are out of date my answer to my team is always "update them!" Anywhere that we're writing documents there are revision and discussion features so it's not like you can irrevocably screw something up, just improve it and let us know what you did. I do struggle with getting people to actually do it though.

Having multiple systems for docs and Slack for follow-up questions is a major red flag. What you’re describing is a billion dollar search product opportunity though. Most orgs don’t have the discipline to have a single source of truth. So you end up with this mess. Run! Or … fix it and then create a company to fix it for all the other orgs with similar data/docs siloes.

I don’t understand why more companies don’t just go all in on Slack as the interface to their knowledge base. There’s tons of integrations to enable it. Every place I’ve worked at with Slack has the standard 90 day retention policy in place which makes it impossible.

One solution to this is to become an oracle at your company. Any time someone (namely someone higher in the company who is responsible for your pay) has a question, no matter how many times it's been asked, how recently it's been asked, how obvious and repeated it is in the documentation, you answer quickly and thoroughly, like a machine. After a year or two of this, you'll be able to ask for any raise, or to work remote, or to even switch to contracting. They won't want to lose you.

This is why I find documentation to be either useless or actively detrimental. Your documentation is the code. Unless you have a dedicated technical writer on the team whose full time job is to work with developers to document their code, it all just becomes an outdated confusing mess immediately.

Obviously this doesn't apply to public facing codebases. But trying to keep an internal codebase documented, other than fully finished self contained library level code, is a sisyphean task.

that's ok. but expect your onboarding to take a really long time.

Agree. If possible the documentation should live in / be generated from the code as well.

I'm not checking confluence, write it in a ".md" file in the repo if you want me to see it.

The way I did this at a company I worked for is that we had a MediaWiki. That's the software that runs Wikipedia. Whenever anyone would ask me a question, I would make a MediaWiki page or add to an existing page and appropriately link the page or entry to other relevant pages and answer the question there. Then I would send them a link to the MediaWiki page. This was super efficient. Whenever any documentation was wrong, I would update it.

Exactly this.

I think that high documentation can work BUT the company has to invest in it in the way that Digital Ocean or even Stripe has done outwardly.

1. Investment - You have to hire at least a few technical writers and librarians to provide training & cleanup functions.

2. Management buy-in - You have to budget for it and encourage it through day one communication (ie. New hire training) and consistently rewarding and recognizing people for getting it right.

One of the biggest issues with large companies is - there could be a lot of documentation- but can you find it.

A lot comes from tribal knowledge/who to talk to/etc.

From my perspective, calling yourself a "high documentation" org leaves a similar impression as calling yourself a "high code" org.

Very much my experience too.

People need to actually learn how to write and maintain(!) documentation, otherwise it’s just a huge chaos.

Rule 1: less (text) is more.

> 5 outdated documents describing the decision making process

Would that place be the U.S. Department of Defense by chance?

Not if there's only 5 documents.

> you should have searched for a specific, obscurely named document in Google Drive, Confluence, or Github

Are people at the same job telling you to check 3 different sources for internal docs? Maybe that is the main issue. Put knowledge in one place.

More specifically: One place that is not Sharepoint.

> no documents about how to actually use the resulting software.

Can all your engineers see all your other engineers' code? It's hard enough to get code to do what it says it does; I've very rarely seen documentation that's correct.

Lack of organization to that degree is an indicator of failed leadership

This sounds so similar to my new place of employment, it has to be the same.

We might be colleagues then.

Reply on news.ycombinator.com