Hacker News Reader

Comment by gavinpc | Hacker News Reader

I recently ran across this piece, "Literate programming: Knuth is doing it wrong", whose author contends that

> the ends are insufficiently ambitious by focusing on a passive representation [0]

After practicing literate programming seriously for over a year on a large project, I believe that our notion of documents themselves is insufficiently ambitious, for the same reason.

Computer-based media should not be limited to the static forms imposed by paper. It's unfair to critique literate programming per se until we have a truly dynamic mode of representation. (edit, yes, technically we have that now, but it's an order of magnitude more work to create. I'm saying that live, dynamic representations should be our tool for thinking, building, and writing about things in the first place, not after the fact.)

[0] http://akkartik.name/post/literate-programming

View on HN

What tools did you use for this?

How long do you have?

This was for a project called "willshake." The system provides a rough overview of itself [0].

I was using Org for documentation, and I decided to try migrating to a literate codebase.

It was clear immediately that I needed a "real" build tool. After some research, I decided that was Tup [1]. Tup is a strict, "functional reactive" build system with extremely fast incremental builds.

So I run the Tup monitor, and everything gets tangled as I write. Great.

But tangling through Emacs/Org was way too slow. So I wrote my own gawk-based tangle [2] that supports the subset of Org Babel's features that I needed, but would tangle any file in 20-50ms.

But usually I want to do something with the tangled code (you know, compile it, run it). So I also define build rules inside of the documents (for example, [3]). I just extract the rules from the documents and feed them to Tup.

Now I can just write whatever I want in a document, and stuff just happens as I write. This includes the removal of obsoleted build targets, which Tup handles. So I can "literally" remove a document file from the project and the feature is removed; return it, and the feature is restored.

But as the system grew, the build started to push 2 seconds. This is because Tup was reviewing thousands of rules whenever any file changed. So I decided (following Christopher Alexander), to treat directories of documents as isolated subsystems.[4]

So I can create features and subsystems with continuous builds ~250ms, using only Org files and directories.

It still sucks. I'm taking a detour to explore dynamic documents, which I think precedes considerations about "programming."

[0] https://willshake.net/about#more-about

[1] http://gittup.org/tup/

[2] https://bitbucket.org/gavinpc/willshake/src/default/bootstra...

[3] https://willshake.net/about/the_stylesheets#sec-4

[4] https://bitbucket.org/gavinpc/willshake/src/default/bootstra...

Thanks for the response. What you've written here seems like the direction I'm trying to explore. However, the corporate proxy server seems to be having trouble with your links (they don't like bitbucket, don't know why), so I'll have to review those after work.

EDIT: Thanks for the tip about tup. That looks like something I could get into. Been reading about it some this afternoon, I'll check it out more tonight or this weekend.

Finally had a chance to review your various links. That's a very impressive project, I like it quite a bit and have now shared your product (willshake.net) with several friends.

Thanks for the links and the resources. My target with literate programming is a bit different than yours, but you've given me some things to think about. I appreciate that.

Thanks, it makes my day to hear that. I think literate programming was ahead of its time in some ways, including ways that maybe Knuth didn't anticipate. Good luck.

Reply on news.ycombinator.com