Introducing sphinx-js, a better way to document large JavaScript projects | Hacker News Reader

Introducing sphinx-js, a better way to document large JavaScript projects

hacks.mozilla.org

90 pointsnachtigall30 comments

I use jsdoc2md[1] and Gitbook[2], with a plugin[3] to integrate the two. An example of docs automatically generated from JSDoc comments: https://scriptare.gitbooks.io/compago/

[1]https://github.com/jsdoc2md/jsdoc-to-markdown

[2]https://www.gitbook.com/

[3]https://github.com/zandaqo/dmd-plugin-gitbook

I’ve long held that there are two types of documentation.

- One, like this and JSDoc, is structured documenting of the code interface. It’s a reference tool.

- Two, is a more free form documenting of intent. Commented source code tools like Docco serve this well.

Both are useful in different situations and don’t replace each other. For 3rd party tools the former is more useful, for internal - the latter.

Unfortunately the parsers for each are usually mutually incompatible and I really wish there was a tool that supported both.

That's what I've always liked about Sphinx: you can get reference docs extracted from source code, but you can also surround them with more in-depth introduction or explanation.

For example, these are the reference docs to a package I used to maintain: http://django-tidings.readthedocs.io/en/latest/reference.htm.... They're entirely extracted from the source.

However, most of the other pages in that manual are written expressly for new users, organized to teach. The magic of Sphinx is that those introductory pages can link effortlessly back into the reference docs. There are many examples on http://django-tidings.readthedocs.io/en/latest/introduction.....

And it doesn't stop at linking; you can also embed extracted docs into the midst of a contextualized explanation. See https://mozilla.github.io/fathom/optimization.html, which is JS code and so uses sphinx-js.

I missed all that power from the Python world; that's why I ported some of it to the nascent JS ecosystem.

It's certainly good at being descriptive but it's still very focused on the reference style interface.

Which is fine of course, but it doesn't really help with the latter case as anything other than a secondary concern. That's great when your primary usage is as a reference but suffers the same problem as JSDoc/etc when it isn't.

But it's a better form of what it's doing, for sure.

You can write a book with ReST. Nobody's going to do that with Javadoc. The documentation generation features are all bolt-ons and not in any way central to how Sphinx works.

JSDoc has @fileoverview, but it seems to be rarely (or insufficiently) used, in favor of overdocumenting functions and parameters. On the other hand, I find that I use @fileoverview far more often. Treat it as a way to explain not the tree in front of you, but where it fits into the overall forest.

Here's a comment of mine from a while ago describing how Mozilla's old "Bonsai" tool worked:

https://news.ycombinator.com/item?id=11381815

I think it is unfair that the article says there is only one other contender.

There exists for example esdoc [1] which is capable of compiling together written documents with generated documentation. I've just implemented this for two work projects with the return of many thanks from my team.

That said I will also check out sphinx-js because we also maintain python projects and already use sphinx for those.

[1] https://github.com/esdoc/esdoc

Author here. I just learned about esdoc this morning; thanks for mentioning it! But it doesn't appear to be much of an improvement over JSDoc for my purposes, as you still get little control over the organization of your content. It is alphabetical reference docs plus a few predefined bolt-on pages (https://esdoc.org/#integration-manual):

    Overview
    Design
    Installation
    Usage
    Tutorial
    Configuration
    Example
    Advanced
    FAQ
    Changelog

You can't add more. And, as far as I can tell (the documentation I can find is silent on the point), those pages are islands, with no ability to call content out of the code. (I'd love to be shown wrong. Did you find otherwise?) However, I do applaud esdoc's better support for more modern JS features!

Some of your assumptions are incorrect. Those names you list are documentation categories, under which you can publish any pages in your own order. The final docs do not actually display those category names, but some have different menu colours by default.

You can also link between written docs and API docs, though it is a bit manual/hacky (you need to know the rendered file paths).

Thanks for clearing that up!

Dgeni has been around for years and (is?) has been used to document the Angular codebase.

I have yet to see anything that actually documents well other than code itself. Comments are _always_ subject to the problem of going stale or being poorly worded. I haven't played with it much, but Clojure's spec looks like a good tool for documenting how functions should behave. That plus a description in the docstring might be the best thing out there right now.

Have you seen Elixir's doctests[1]? They're docs with a code example that gets run by your tests, and if the example doesn't work, your tests fail.

[1]https://elixir-lang.org/getting-started/mix-otp/docs-tests-a...

Python also has a similar concept: https://docs.python.org/2/library/doctest.html

Sphinx also has additional doc testing methods that let you test the code examples in your actual docs: http://www.sphinx-doc.org/en/stable/ext/doctest.html

I was surprised (and pleased) to find out that Rust has doctests as well.

I haven't seen that before, but I really like it. I've thought before that it'd be a good idea to include tests adjacent to the code they're testing before, but I'm not sure of a good way to do that, particularly in Javascript.

IME it's an OK way to show examples which are validated (and thus don't get broken as the API changes), but it's really not a good way to have actual tests. Terrible debugging experience aside, actual unit tests just make very little sense as part of a documentation.

The idea is to provide a DRY way to ensure that the examples are correct. Few things are worse than a wrong example.

That's a fine idea and exactly why doctests exist but I am specifically replying to a comment talking about tests. Not testing examples, having actual tests included in the documentation. Which, again, experience taught me is a terrible idea.

They're really only for documentation and the simplest of single function unit tests.

I can see that working for a functional language. But how do you specify the side effects?

You would have to assert equality on the value of the data/file/whatever being mutated.

Here's an example:

##Examples

  iex> user = Repo.insert!(%MyApp.User{id: 1, some_state: "whatever"})

  ...> {:ok, user} = MyApp.Users.DataTransformer.transform_with_effects(user)

  ...> Repo.one(where: MyApp.User.id == 1).some_state

  "new state"

On subject of going stale, there is valid-jsdoc ESLint rule that will ensure doclet comments are kept in sync with ever-changing signatures.

http://eslint.org/docs/rules/valid-jsdoc

esdoc also includes a linter which does the same

There's a reason The C Programming Language is one of the most influential books in the history of computer programming...and probably still worth reading today even if a person is not likely to become a C programmer. Its prose describes and explains the code and helps a person understand it. On the other hand, the prose is not comments. The prose is literature.

Or to put it another way, the way to learn how to use Clojure's Spec is not from the names of functions in the Clojure Spec source code. It's from reading blogs and watching videos and listening to podcasts and reading the documentation. And the bar for Clojure Spec is pretty low. It just has to be better than JavaDoc.

Completely agree. There is no reason (outside of some kind of coding challenge) for code bases in the web domain to contain logic that is so obtuse due to hacks or unavoidable complexity that spending more time to document functions than it took to write them is the correct use of business resource.

If you like Clojure.spec, you'll love a proper type system. ;)

More seriously, keeping comments up to date is hard, but there's no alternative if you want to maintain a public API.

That's the whole point of doctests: https://docs.python.org/3/library/doctest.html

Something like this has been around a while in the enterprise world. Take a look at Cucumber for Java or the C# port SpecFlow. The test cases are the documentation.

Agree. This is why Typescript/Flow became so popular, because they break the build if the "doc" isn't right.

Reply on news.ycombinator.com