The CWEB System of Structured Documentation | Hacker News Reader

The CWEB System of Structured Documentation

www-cs-faculty.stanford.edu

60 pointsweeber13 comments

Open articleView on HN

See also NOWEB, a system that developed the ideas of CWEB. Wrote my dissertation using it.

https://www.cs.tufts.edu/~nr/noweb/

Fraser and Hanson wrote a compiler as a book, using noweb:

https://a.co/d/4ctR3vW

I studied CWEB when I did computer science in the early 1990s (eek!).

While it is a way to document and compile your programs from a single file, it still contains the problem of where the story of a program should happen, and in keeping the documentation and code in line and up to date with each other.

It should also be understood that the documentation is for other programmers and not users, since the printout of the document includes the code in-line.

I think that underlines a major point, though. CWEB is specifically to document the program for reading. It is not user documentation. It is not a design document of the intent of the system. It is documentation of the code, specifically.

It can be difficult, because the assumption is emphatically not that the reader in unfamiliar with the target language. To that end, much of the code you find in CWEB will still be using idioms of the target language. Can still be somewhat jarring if you are not expecting that.

Counter-example.

Have used literate programming for a real-life freelance project a year ago. Am using it for both my private projects and current freelance projects.

The manual can be broken down into four parts or volumes depending on size:

  I. User's Guide
  II.  User's Reference Manual
  III. Maintainer's Guide
  IV. Maintainer's Reference Manual

Don't get this wrong, updating the screenshots in the User's Guide is annoying and a pain. Over 60% of my time go into writing soft documentation for the users. This seems to be the best way to have hard evidence of the gathered requirements. It shows what the user actually faces. The user stories write themselves. This is the place to put assumptions and data-flow requirements that the human being in front of the tool has to know. The customer last year loved it.

Testing becomes easy. Just follow the pictures in the User's Guide. That's your checklist. In last year's project we immediatly found differences in using the application on iOS and Android, just because the customer was on iOS and could play around with everything immediatly [0]. The first working prototype was done in a week. Iteration was fast in a tight loop with the customer. Basically, live coding.

Source code, entity-relationship diagrams made with Dia diagram, data-flow diagrams and even some math typesetting go into the Maintainer's Guide. All those hairy assumptions forced upon you by third party libraries, external APIs, and external ABIs go in big fat red boxes. Ongoing maintenance check-lists, build steps, and deployment issues also live here.

LaTeX makes it look beautiful. It is, honestly, a joy to read and nice to look at. Just like with research papers decent pictures make it better.

The reference guides are basically lookup-tables. There can even be an index and a glossary.

The PDFs can even be stored in git(1). I know, I know, don't store binaries in git. But it archives the project well [1a,1b]. You already have the screenshots. You can pull old versions out. All the hidden footguns can be called out in the text.

  "It is better to solve the right problem the wrong
   way than the wrong problem the right way."
      -- Richard Hamming as quoted by Doug McIlroy in
         Eric Raymond's The Art of Unix Programming. [2]

[0] [James Hague: Documenting the Undocumentable](https://prog21.dadgum.com/161.html)

[1a] [Archive Your Old Projects](https://arne.me/articles/archive-your-old-projects)

[1b] [Archive Your Old Projects (HN)](https://news.ycombinator.com/item?id=38239358)

[2] [Solving the right problem](https://www.catb.org/~esr/writings/taoup/html/ch01s04.html)

Do you have a link to your repo where you've done this?

No you can combine both, which is the preferred way. As an example (although it does not use CWEB, but its own code) LaTeX's .docstrip/doc/dtx system does that. It generates the code, a driver and documentation both for code and user. It also produces an index automatically. You can use it to write your Javascript code if you want. As Wirth or maybe it was Hoare used to say code is literature as it is meant to be read by humans not only computers.

Fair. Though I think it is a safe to say that the intent is to document the software moreso than the use of it. Mainly going off of all of the programs on Knuth's homepage. TeX and METAFONT are both the same. The literate program is distinct from the user manuals.

CWEB spawned several other Literate Programming systems, the main differences being in programming languages supported as well as how the documentation was formatted.

My favorite is nuweb https://nuweb.sourceforge.net/ which allows for LP in virtually any language and provides documentation in LaTeX. The main trade off being that with wide language support the code sections are as-is, without special formatting. Since mostly people write the code sections to their own tastes already this is not a significant problem.

I use Noweb <https://www.cs.tufts.edu/~nr/noweb/>, which is, I believe, very much like nuweb. It is also LaTeX-centric, and also allows code sections to be written in any language. Within code sections, instead of depending on Noweb for code formatting, I use a syntax-aware editor. I configure Vim to recognize the language within each code section. This means that, in Vim, I get formatting and syntax highlighting.

Vim is also able to export syntax highlighted code as HTML markup. Because Noweb is modular and supports plugin filter stages, it might be possible to write a Noweb filter stage that calls Vim, translates the markup to LaTeX, and adds syntax highlighting to the document. This is something I haven't tried.

I find it surprising that nobody talked about Jupyter so far. I still think that Jupyter is an overly simplified realisation of cweb's vision.

isn't the first problem of documentation is keeping it in sync with the source code? Time and time again people come up with "better ways of documentation" while neglecting this basic problem, what problem do they think they're solving?

Yes, but the closer the documentation is to the code, the easier it is to keep up-to-date. I've found anecdotally that other developers on my team readily update doc-comments when method signatures change, but update out-of-file documentation less readily (such as a markdown readme document).

Structuring the documentation as part-and-parcel of the code helps give the documentation a fighting chance of staying up-to-date over time given without requiring all developers on the team from being religious about updating docs (which I find I need to enforce at the PR stage for those repos with external readme files).

Reply on news.ycombinator.com