Comment by falcolas | Hacker News Reader

Comment by falcolas | Hacker News Reader

Parent Full thread

> User testimonials and evidence of past performance in real development situations

Please, not in a README.

My problem with documentation remains: It's always out of date, and frequently contradictory. I'm trying to work with Kubernetes right now, and by this "maturity model", it's a Level 5. That doesn't actually provide any real help though, since k8s is moving so quickly the tremendous volumes of documentation which make it so attractive to managers and leaders just can't keep up.

What arguments should I be providing to kubelet? Depends on whether you're reading the code, the admin guide, or the getting started from scratch guide. None of which, by the way, reference the version of k8s you're going back to.

The API docs generated from the code are a bit better, but they were written as part of the code, and the descriptions are incomplete and confusing without the context of the code (separate rant - that codebase is split between so many repos you'll go slightly mad trying to learn what is where if you're not already familiar with it).

In the end, we always end up going back to the code; even k8s documentation acknowledges this by constantly linking back to GitHub.

I just had an idea, but more in relation to documenting source code: what if there was a tool sort of like gofmt in that it has to be run in order for code to successfully compile, but instead of formatting, it actually checks if some or all of the variable names in functions or objects appear in some corresponding comment (like Python docstrings [0] or javadoc comments [1]).

I just came up with the idea and the first (bad?) heuristic now, and I didn't check to see if tools with these characteristics already exist. It would be bad initially [2] and it would be onerous for sure, but it would be a start.

[0] https://www.python.org/dev/peps/pep-0257/ [1] http://www.oracle.com/technetwork/java/javase/documentation/... [2] https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_fu...

In rustdoc, documentation examples are compiled and run whenever you run your tests. It's very helpful for this kind of thing.

https://doc.rust-lang.org/stable/rustdoc/documentation-tests...

Thanks for the example!

To be completely clear, this is what I meant:

Suppose this is the function we want to document (using examples modified documentation tests:

  fn foo {
    let x = 5;
    let y = 6;
    println!("{}", x + y);
  }

Suppose we have a documentation requirement/heuristic of requiring the comment to have all the variables that are declared in the body of any given function. Then a comment like this would pass because whatever compilation step /processor sees that `x` and `y` both appear in the comment.

  // Prints x + y

This would fail because `x` is not found:

  // Prints z + y

This would pass but is not useful/misleading/wrong.

  // y - x FTW!!!

This example is trivial and only uses the one simple heuristic such that it is both onerous and useless, but it would force the writer of the code to also write comments in accordance with said heuristic. In this particular case, if someone modifies the variable name, adds a new variable, or removes and existing once, then the compiler would force that individual to make a necessary change to that particular comment such that the comment cannot become stale (but the heuristic is bad, so it could still be useless :) ).

I would argue that externally documenting declarations inside the function is not particularly helpful.

Just wanted to chime in and say that as a Go developer of almost 5 years now, running into Rusts' compiled examples has been quite cool.

The cool thing about these is that you can actually (depending on the situation) make your examples your tests.

What makes them so different from godoc's examples?

Python has something similar:

https://docs.python.org/3/library/doctest.html

For software with a command-line interface, perhaps it would be useful to do some testing at that interface, perhaps using BATS:

https://github.com/sstephenson/bats

That is also automatically tested, but describes the actual user interface.

Nice! Python has doctests which serve a similar purpose:

https://docs.python.org/3.6/library/doctest.html

In the Python world, about the best you can do is enforce the existence of API documentation. Most Python linters can check that every module/class/function/method has a docstring, though of course they can't enforce that it's up-to-date or useful to a human.

Enforcing useful up-to-date documentation requires cultural setup work rather than technological setup work.

What about documentation, even "how-to use this product" documentation, that's generated from the code's comments directly?

So say you have a function that essentially implements the "--file" flag: what if the developer put a comment above this function explaining the function's purpose but also the documentation for the admin's use case, which would be setting up the solution?

I believe that if such documentation existed it would be a lot easier to update it when the code is changed (provided the API changes) and have it propogate through to the docs. The comments could also be three tiered: API docs, admin docs, user docs; each would explain to the relevant user how-to make use of the code at that point.

Obviously you could omit user/admin comments for functions that aren't "public facing", such as internal helper functions. But anything that's directly interacted via a CLI flag or HTTPS endpoint could have the documentation right there by the code handling that interaction... or not if it's not relevant.

See "godoc" as an example of this: https://blog.golang.org/godoc-documenting-go-code

The code truly does become the documentation at this point, and acts as a central point of truth.

This. There's an almost inevitable orchestration involved in maintaining good documentation for a fast-moving codebase. Tools like Swagger[1] are good for automatically generating API documentation, but beyond that there is little help for maintaining generic README-style documentation.

How do large, fast moving projects like the linux kernel manage this?

[1]: https://swagger.io/

Linux delegated its user-facing documentation to third parties pretty much from the beginning.

Even for those things which have documentation in Documentation/, there is no formal or informal rule that if you make a change which causes existing statements to become false you have to update them.

Which doesn't reduce the usefulness of the linked document at all.

If your project is bigger, README probably should not be a full documentation of the project but link to the actual documentation.

Reply on news.ycombinator.com