How developers use API documentation: an observation study [pdf] | Hacker News Reader

How developers use API documentation: an observation study [pdf]

sigdoc.acm.org

102 pointsfanf221 comments

I write the Chrome DevTools docs. "Empathy" is a buzzword in technical writing. Recently I wrote a post [1] imploring technical writers to start recording themselves as they use documentation. I want to ask you all to consider doing the same and sending me the footage. My main argument is that the best way to actually develop empathy is to watch lots and lots of people actually use documentation to get stuff done. I've already learned a lot just by watching the videos [2] of me build an Angular app.

[1] https://kayce.basqu.es/blog/empathy

[2] https://www.youtube.com/playlist?list=PL6UItmiYqJhMr_uYG2DkV...

> "Empathy" is a buzzword in technical writing.

We should use a term that is more clinical in technical writing. The "empathy" we're aiming for is fully understanding the user experience. That is a correct use of the word, but the popular usage is closer to "sympathy," and often people are using it to indicate they care about treating people well in a broader moral sense. (E.g. "do unto others" is morally empathetic.)

The difference is that being empathetic morally is an end in itself, whereas being empathetic technically is a means to an end. Dark UI patterns, for example, are very empathetic in the technical sense in that they use understanding of the user to manipulate them.

"I have a problem, ok I will go to the docs and use the search function to find things that may solve the problem"

I can't imagine anyone else does anything but this aside from initial foray into a API

Over-reliance on search may be a sign that you are unfamiliar with the general patterns and background knowledge of how this API is supposed to be used. It is a shortcut, a very useful shortcut nonetheless but it definitely shouldn't be the only way to do things.

I would go as far as saying that if your API requires familiarity with "general patterns" and "background knowledge" you probably have done a bad job designing it.

Your API has a purpose. It does things. I know what that is, that's why I came here to use your API. I can verbalize what I want your API to do for me, and that is what I will search for. I will not look at your whole documentation, I do not care about your general concepts. All your ideas about architecture and design stand between me and my goal. I want you to apply the principle of least astonishment. If there's something non-obvious that might cause issues, mention it in all the places where it applies. Yes, this goes against your urge to not repeat yourself. Do not give in to these urges.

Sometimes people design APIs with great flexibility in mind, requiring you to assemble a great many parts in the right shape to achieve your goal. This is almost certainly not what the user wants. If you intend to design such an API, use the "layered" approach. At the top, there should be the most common functionality, as obvious as possible, with as little configuration as is reasonable. This is the layer that almost everyone will need - and not more. This layer is implemented using the more flexible layer below.

That's sort of presumptuous. For example, I recently needed to create a ORM w/ SQLAlchemy that consisted of animal/{dog,cat} style inheritance. I googled it and came across this wonderfully written page:

https://docs.sqlalchemy.org/en/13/orm/inheritance.html

I've been using SQLAlchemy for about 3 years now and I have used it to make my life so easy that I feel bad for other people who struggle with SQL.

Does this mean I am not familiar with it or that I know exactly what to look for?

Food for thought for you, perhaps.

Not to mention any given user may only use a handful of different aspects of SQLAlchemy, and the documentation is seemingly endless. You could spend a year just reading docs on parts of SQLAlchemy that you'll never use.

Also can look at a series of searches a person makes and how long they spend on each page. And the things searched for are things that could have perhaps been described better or linked to from the original page.

I feel like this is a bit light on data, since it's only 11 people from 3 companies using 1 (non-English) website.

The takeaway, though, is consistent with what I've seen... people like code examples! Most people will scan until they see one, and start there. The heavy focus on API Reference in this study makes me think either the tasks were too specific, or the tutorials weren't comprehensive enough. For a good documentation site, most use cases should be covered by topical guides. Going to the reference guides to learn an API is like reading a dictionary to learn English. There's not a lot of context.

> Going to the reference guides to learn an API is like reading a dictionary to learn English.

That doesn't necessarily have to be the case. Back when I was at an IoT startup I structured the reference docs based on the order that a developer would typically use in order to get common tasks done. E.g. call this method to get the device's identifier, and then here's a list of methods that let you do stuff with the device. Chris Bush presented a similar idea [1] at this year's Write The Docs conference. His main argument is that the organization of reference docs is usually implementer-oriented. E.g. alphabetical. In many cases it may make more sense to have a user-oriented organization, one way or another.

[1] https://youtu.be/4OtdFEp88VI

> His main argument is that the organization of reference docs is usually implementer-oriented.

This is absolutely the biggest (and easiest) mistake people make when designing APIs. Force developers to use their own APIs as end-users - quality will improve significantly.

This reminds me of the Autotools duality: most developers hate autotools because of its unnecessary complexity but users (ie. those who want to compile the source) like it because of its flexibility (most autotool-based projects build pretty much on anything that smells like Unix), configurability and use of standard tools (shell, make) that are already installed (pretty much every other build system - like cmake, premake, meson, scons, etc - need itself installed on the user's system).

So you get developers hating it and users (especially admins and distro packagers) loving it (well, packagers who have to build a lot of stuff sometimes dislike the performance).

Unfortunately, there's not just one autotools but at least two major variants: libtool and direct, with 3 common incompatible versions in the wild: 2.1, 2.6, 2.13.

Packagers have some trouble getting fixes to these build systems to stick. Cue Gentoo autotools.eclass with automatic application of generic set of fixes for all the brokenness of the ages.

AFAIK this is a problem only if you want to regenerate the configure scripts. As a "user" (and packager) you should use the pregenerated configure script provided in the release.

The examples also do the important job of filling in gaps that the reference docs may accidentially leave - one common one for http APIs I've found is how non-string query params are encoded (eg. this value is a list: do I give it multiple times? Do I comma-separate? Or worse: is this enum value a string or an integer?).

What's especially fun is when the "code examples" for a http API use a client library which hides the actual http API, so they aren't examples of using the API at all...

> Going to the reference guides to learn an API is like reading a dictionary to learn English. There's not a lot of context.

Exactly! Examples are usually the most useful thing in the documentation.

Based on my experience, people go first to the examples and then read the description.

I guess that's also why stackoverflow is working so well as it is basically a large library of examples.

I’ve never understood why more apis don’t hyperlink they their code samples.

It would massively improve discoverability.

That and give a single large actual example of interacting with the API.

That said I’ve always preferred reading code using a library over reading docs for the same library.

It took me a bit of time to get used to them but I am really starting to love the vim docs - I love that I can :help split and then get loads of info of all the possible things I can do with a split for example

Mods: please add "How" back to the front - it's needed for the sentence to make sense.

Yes. Done.

Thanks

Reply on news.ycombinator.com