Hacker News Reader

Comment by jashkenas | Hacker News Reader

We've actually talked about adding this to CoffeeScript in the past, as a regular language feature. (You've probably already read the ticket.)

The reason we don't is because good tests are very rarely the same thing as good examples. Good examples often elide bits of code that aren't pertinent to the API function being demonstrated ... and good tests often involve the edge cases that are poorly suited to learning examples.

While testing your examples is all well and good, it's also limiting -- you can no longer write:

    makeRequest({
      url: 'http://example.com/endpoint',
      data: ...,
    })

... unless, come to think of it, you had some sort of Yada-yada operator that served as a silent no-op.

View on HN

FWIW, doctest.js briefly supported CoffeeScript, back when the format was essentially language-neutral:

    $ makeRequest({
    >   url: 'http://example.com/endpoint',
    >   data: 'foo',
    >   success: Spy('makeRequest', {wait: true}),
    > });
    makeRequest(...)

The example and output was parsed essentially by regexes. That was kind of okay, but I'm much happier with the new comment-based format. But that depends on parsing the language itself, so it's nontrivial to add other languages (before I only had to replace eval).

Still, supporting CoffeeScript shouldn't be a big task, it would just require abstracting out some of the parsing, and having a way to select the language (probably just another CSS class on elements, and/or the page body). The only parsing doctest really cars about is chunking the text by comment.

In regards to abbreviated examples: you can write your test in one big chunk, and I like to do this when I'm really testing and not explaining – doctest.js is not intended to just be a documentation tool. But if you are composing docs, then doctest will only test things that are explicitly marked to be tested. It's not as strict as Python's doctest, because it's using essentially invisible information to determine what's a test (a CSS class). You can also have it test things that aren't visible in the docs themselves, but are necessary to create a usable environment for the tests.

Of course a lot of things, like makeRequest, are only interesting if you handle the async aspect of the code. Spy() is handy for testing, but will look peculiar in documentation. It occurs to me that for documentation you might want:

    makeRequest({
      url: 'http://example.com/endpoint',
      data: 'foo',
      success: function () {
        print('I got a response:', this.status);
      },
    });
    // (later) => I got a response: 200

Where "(later)" tells the test runner to wait for at least something to be printed out.

I'm waiting eagerly for the Literate CoffeeScript support to land.

My main concern with it is that it forces you to write the document in the same order you want the code to be extracted, which may not be the best order for explaining things. That is why classic Literate Programming tools like noweb[1] allow you to name the chunks of code and then arrange them into the generated files as you wish. You can see an example of this in action when I'm assembling noweb.php[2].

Also, agreed on the point that not every piece of example code is suitable for running as a test, so you'd optimally want some sort of annotation for that. Github-flavored Markdown[3] has the code block fencing syntax that would allow metadata like this. Something like:

  ```coffeescript
  # Just a regular, non-testable example
  ```

  ```coffeescript;test
  # Example that is also a test
  ```

Actually, this same approach could be used for working with named chunks like noweb does.

1: http://en.wikipedia.org/wiki/Noweb

2: https://github.com/bergie/noweb.php/blob/master/README.txt#L...

3: http://github.github.com/github-flavored-markdown/

    > My main concern with it is that it forces you to write the document
    > in the same order you want the code to be extracted, which may not
    > be the best order for explaining things.

... Yep -- this is probably the main complaint raised when talking about tools like Docco or Markdown-as-source-code as "literate programming". But, it's a concern that I believe is entirely outdated. Modern dynamic languages make it easy to sequence your code as you like. The methods in a class may be listed in any logical order, helper functions can be listed in an appendix after the functions that make use of them, and so on. I find it hard to imagine an example where changing the order of the codebase would make the prose version more readable -- and wouldn't also make the code version more readable as well.

I think that, unfortunately, the tangle/weave paradigm is a large part of why literate programming is barely used -- it introduces an entire level of build-time complexity that doesn't have any direct effect or benefit on your code. Here's hoping that the simplicity of having a file that is at once both executable code and publishable markdown is something that feels more usable to more programmers.

Good tests aren't necessarily good examples, but good examples are good tests. In other words, good examples are a subset of good tests.

It's also good practice to test your documentation. Far too many examples in documentation have subtle errors that could easily be caught by regularly executing them.

Reply on news.ycombinator.com