Hacker News Reader

> It isn't.

Well, whether it is or isn't depends on how good the docs are and how well you write them. Yes, in many cases it can be easy to forget if the documentation is not woven in well enough.

> All public APIs should have documentation, even if you believe it's obvious what they do.

As a note part of the function of such documentation is to establish standards for what is acceptable in terms of expected input and output handling. What this means is that if documentation defines the code contract, then the first thing you look at when debugging is the API's documentation. Then, if it matches what you are doing, you might dig deeper.

What this gives you is not debugging by comments (something K&R rightly hated) but asking which side the violation of code contract is on. If the documentation doesn't match what you are doing with it, then the violation is on your side. If it does, then the violation may be on the API's side. The goal here is to define where changes can most productively be made.

> Because there's other code in the wild that relies on the behavior that you promised.

That's exactly right. More specifically the API documentation is the promise.

> If you're changing it so often that updating the damn comment is an issue you either document implementation details that don't belong in API documentation or your API is unstable crap and nobody should be using it.

The thing is it took us a long time to get our documentation approach right in LedgerSMB. It was a struggle that really only I think reached something I am happy with 5 years into the project. A lot of our public SQL API's are not documented actually, because they are dynamically discovered at run-time and are minimalistic (and consequently the developer contracts far more vague than the API conventions, so it isn't always clear what belongs in the documentation since it is all dynamically looked up anyway), but our Perl code is very well documented and I am very happy with that.

For the SQL though, it's written with documentation generation scripts in mind and therefore the question is what you can document on top of what is already there in the system catalogs.

You can change an API without removing functionality. Sometimes you want to add functionality. Or sometimes the context around the API changes. Like for example, a python API might not function the same in Python 2.5, Python 2.7, and Python 3.

All I know is, I help maintain a very large set of public APIs that my team is very resistant to changing, and yet somehow the docs are still out of date.

Is that because you're not updating your docs?

> Complaining that API documentation gets out of sync with the code is like complaining that unit tests break when you change the code.

But that can be a valid complaint as well. There is an undeniable cost of maintaining tests, and it shouldn't just be taken for granted that the cost is worth it.

In both cases, you shouldn't be documenting and testing the internals if your functions, but the code contract, i.e. that it does what you have agreed with other developers that the function will do.

The question is not whether to test or whether to document, but what to test and what to document.

I don't disagree that public APIs should be well documented. However, I believe it's very easy to forget to update the comments. Especially in scripting languages without type safety. There is little to remind the programmer that arguments were added, types were changed, argument order was modified, behavior was changed, return value was changed, locking behavior was changed, memory allocations were changed, etc. Obviously unit tests should help catch many of these changes. However, it's still up to the coder to remember to change the comments. When you're behind on a deadline or you're juggling 50 different API changes because you're still alpha I'd say it's pretty easy to forget to update a comment.

See http://api.jquery.com/jQuery.ajax/ if you want an example of on API that could easily have a few typos in it. Who is checking to make sure it's 100% in sync with the code 100% of the time? Not trying to say this example has bugs in the docs but there's a LOT of behavior described that could be out of date.

But the solution is to make sure that the documentation is of a sort that is useful in determining where to fix things. Again, if you have a problem with a function call, the fist question should be "does your call to the function match the documentation?" If it doesn't the first thing you do is change the call to match.

Now sometimes one comes to the conclusion that an API is broken, so you have to modify the comments, and then modify the code, but there is a reason to do it in this order.

When you modify the comments you are modifying a set of promises you have made to other coders. This allows you to think through how this change is going to work and how it will affect other code out in the wild. Then, when you modify your code, it is going to be better.

If you're forgetting to update the comments frequently enough to be a problem, you're probably forgetting to update the code that consumes the API too. That quality in general falls by the wayside when you're crunching is no reason to abandon quality altogether.

Forgetting to change the code that consumes the API means that things don't work, which is visible right away. Forgetting to change the comments will be visible gradually and cruft accumulates. Also, sometimes comments wind up being non-local and just aren't noticed. I agree that in some senses it's "no excuse" but that doesn't mean it won't happen. Documentation that can break visibly when things change is better than static documentation.

> Forgetting to change the code that consumes the API means that things don't work, which is visible right away.

Oh how do I wish this were the case!

> Documentation that can break visibly when things change is better than static documentation.

Agreed, but I would not call your average run-of-the-mill unit tests "documentation", nor can all documentation can be programatically tested.

> > Forgetting to change the code that consumes the API means that things don't work, which is visible right away.

> Oh how do I wish this were the case!

It's certainly not always the case, but it's a whole lot more likely to be the case than for unchecked documentation.

> > Documentation that can break visibly when things change is better than static documentation.

> Agreed, but I would not call your average run-of-the-mill unit tests "documentation",

I think "is it documentation" is probably more of a spectrum than any particular threshold, and run-of-the-mill unit tests probably do fall on this spectrum though I'd probably agree that they're not particularly far along it (though that surely varies with the habits of those writing the tests).

> nor can all documentation can be programatically tested.

As a practical matter, that's certainly currently the case - tooling is not set up for testing documentation, and there are things we'd want to check that would be hard to check in any event. Theoretically also, there are certainly properties that can't be statically demonstrated. I'm not entirely convinced that there's nothing we're interested in that couldn't eventually be got at for the programs we care about, though it's certainly a possibility. Regardless, it seems an ideal worth pushing towards, and if tested documentation is interwoven with untestable documentation such that some conceptual locality is preserved it's less likely (though absolutely still possible, to be sure) that you'll forget to update the other when you're forced to update the one.

Don't you see the difference between changing a comment that doesn't break a build and breaking code that breaks the build because a test fails?

Which are you suggesting is worse?

Well, the point is that it's EASIER to make hard to detect errors in code comments than it is to break tests.

>> When some coder changes the function, it is very easy to forget to update the comments

> It isn't.

Yes, it is. Have done it myself loads of times.