Literate Programming Matters

Direct link to the discussed Cafe au Life, for those that haven't already seen it: http://raganwald.github.com/cafeaulife/docs/cafeaulife.html

And in terms of the gist of Reg's essay, I think the single line that best cuts to the heart of it is: "[...] while David presents the concepts of literate programming and elegant programming as a dichotomy, I think they're orthogonal." Bingo.

@raganwald: I'd be curious to hear more about what sort of "literate programming tool that transforms the source directly" you were hankering for. Not CWEB style?

Literate programming can be done in any language. Here is an example in HTML: http://axiom-developer.org/axiom-website/litprog.html

In addition to the benefits mentioned there are at least 3 others I can site.

First, since you spend time explaining why you are writing the code and your thoughts on the design and implementation you naturally discover edge cases, missing cases, and bugs. As a result, the quality of the code is higher.

Second, if you program in a team that does code reviews, the team can see your approach to the solution and the reasons. They can critique your work at a more profound level. If the code review happens before accepting the change commit, the quality of the code is higher.

Third, code lives. Sourceforge is a gravesite of hundreds of thousands of programs that have died because the authors are no longer maintaining the code. New users are confronted with a source tree of tiny files which they are unable to understand and therefore unable to modify and maintain.

I have the "hawaii test" criteria. If you can hire a new developer, give them your program, send them on a 2 week, all expense paid vacation to hawaii, and when they return they can modify and maintain the program as well as the original authors, then you have a fully literate program.

As for the question of debugging, I find that it is no different. Generally the source of bugs are the same (e.g. using copy/paste and failing to properly fix the copy).

Literate development, for me, takes on two different styles depending on the language.

In a language that allows a read-eval-print loop, like lisp, it is trivial to work in emacs with a command line in one buffer and the literate sources in another buffer. You just point at the changed code and evaluate it immediately in the other buffer. It is very productive.

In a language like Java that is a pure compile environment I create a makefile that extracts the code from the literate document into the proper com.foo.baz.... source tree, compiles the code, and runs the test regression suite. This generally takes less than a minute or two for most reasonably sized Java programs. So I make a small set of changes, save the buffer, run 'make', and see if the tests pass. For TDD programming I find this works very well.

My last large program was 60000 lines of Lisp in a latex literate file. Tex'ing the file with all of the literate documentation generated 6000 pages. The technology scales quite well.

Learning literate programming is like learning lisp. You keep wondering why anyone would program this way until you suddenly "get it". Once that happens you wonder why anyone could program any other way.

Tim Daly

One thing I really like about literate programming even beyond the extensive "write about what you code" aspect is the availability of sub-procedural refinements. Quite often you write code where everything in a "paragraph" pertains to a specific matter, often preceded by a comment line saying so (e.g. "// Initializing database"). With literate tools I can just move that to a separate block and have a better overview. As this is mostly out of a poor need for visual organization, factoring everything out into small sub-routines and then taking care of scoping and passing needed parameters often seems like a crutch to me (especially in procedural code). Folding editors provide another option for this, of course. (Well, you also could use gotos. But no one ever did that. Especially not me. No, officer, never.)

If I remember correctly, I once had two C preprocessors that did just that, without the rest of the literate programming features. (Found a paper for one of these: http://page.mi.fu-berlin.de/prechelt/Biblio/refinement.pdf)

Rant: Github and Google Plus are not blog platforms!

Here, even when I have the userscript[0] to show the full domain names so I know when something is being posted by a user of a company instead of the company itself, it still just shows "github.com" leading me to think it's Github and crew talking about literate programming.

(I'll admit, I saw it was raganwald and that gave it away, but still).

[0] https://github.com/johngibb/Hacker-News--Show-Subdomains

i am always thinking how to make my code readable, if not fun as novel, at least it should read like a prose telling stories. For those lit programming solutions on the web, nothing feels right. For me, it is ugly to see the comments when you are working with the code daily, especially when debugging. at last, i decide to keep a file for document everything, arch, bugs, decisions, stories,... The document grows over time, and then i will try my best to keep the code shorter. of course there are inlines comments, but most just little sweet ones. if you are going to print the code on paper, mass comments is not going to help.

I do wonder how hard it is to debug literate programs

This is one of the better examples of "literate" programming I have seen. I do have a couple of criticisms though.

The first criticism is perhaps more of literate programming, the concept, than this example. I personally find it difficult to read when each line of code is disjointed by comments. I guess I prefer the chunk size to be larger; a coarser granularity.

Some of these comments really seem unnecessary. for example:

  # Export `Cell`
  _.defaults exports, {Cell}

I think that is obvious enough that the code is "exporting" 'Cell'. I couldn't tell you why though.

My second criticism is that it seems comments are all too often of the "what" variety. Simply translating the code to english. That's not really very helpful. Once someone has some grasp of the programming language being used, the "what" is right there in the programming language. No need to restate it in another language.

What is helpful is the "why" of a chunk of code. I can read plain as day what it is doing. But why is the code doing that? Why was it written? Why is it necessary to do this particular thing? To me, at least, that seems much more helpful.

I feel my commenting has gotten much better since I started paying attention to when I was writing a "what" comment, caught myself, and wrote a "why" comment instead.