Interactive Docs with Markdoc
stripe.com
stripe.com
Now, it seems that 99% of documentation tooling advertised is Markdown based; and e-v-e-r-y time I see this on HN I wonder why AsciiDoc[0] isn't more prevalent than Markdown.
AsciiDoc is older than Markdown and out of the box supports things that Markdown doesn't, which has led to a proliferation of Markdown flavors that remind me the web browser experience of the 2000s.
It is Stripe's documentation that led me a few years back to look into replicating their documentation, particularly the interactive part of it. While I wasn't able to find any tool out there that did this out the box, that's how I found about AsciiDoc in the first place, and then Antora.
This tool, Antora[1], is based on Asciidoc and allows you to create amazing documentation websites. A while back I experimented and wrote a PoC of a plugin for Antora that made my documentation interactive.
I also believe that AsciiDoc is what O'Reilly authors use to write books (edit: it is one of 3 languages they [2]use: AsciiDoc, HTMLBook, or DocBook XML)
I wish AsciiDoc was more commonly used.
[1]: https://antora.org/
[2]: https://docs.atlas.oreilly.com/index.html#what-is-atlas-DNIR...
AsciiDoc gets a lot of things right, but it also has a lot of syntactic complexity and idiosyncrasies that were challenging for our documentation contributors. In documents with structural complexity, it becomes unwieldly very quickly. For example, having to vary the length of the delimiter line when nesting delimited blocks[1] is pretty arcane and leaves a lot of room for errors to creep in when reordering pieces of content. Even though AsciiDoc's extensibility model was compelling to us, it's really not designed for the kind of deeply-nested hierarchies that we needed to be able to express things like our integration builder[2] UI.
Anyhow, we have a section in our FAQ[3] that provides more context about why we didn't choose AsciiDoc. I remain a fan of AsciiDoc even though it didn't fully meet our needs, and Markdoc was definitely influenced by the things that we thought AsciiDoc got right.
[1]: https://docs.asciidoctor.org/asciidoc/latest/blocks/delimite...
> but it also has a lot of syntactic complexity and idiosyncrasies
I don't think that's true; for basic usage, which is probably 90% of what people use, Markdown and Asciidoc syntax are basically the [0]same (AsciiDoc is also compatible with Markdown's syntax)
For advanced use, Markdown syntax has to be augmented via one of the many "flavors" (i.e. no real advanced features out of the box) and these flavors usually mean typing raw HTML directly in the documentation.
Asciidoc advanced syntax on the other hand is available out of the box for whoever wants to use it, but you don't have to use it.
When you take away the include directive you have a less elegant syntax that is harder for contributors to read, much less adopt. Add to that a very small community and a lack of critical tools. Most glaringly, there is no linter for AsciiDoc.
AsciiDoc just doesn't have advantages over Markdown (including partials, as Markdoc and other tools used by IBM and Microsoft allow them). Meanwhile Markdown renders everywhere, and it's highly legible as source.
Does Markdown support partials/includes, and does Github honor them? I don't believe it does... (I tried googling about it, didn't find anything, but I could be wrong) so not sure how this can be used as an argument against asciidoc.
> When you take away the include directive you have a less elegant syntax that is harder for contributors to read, much less adopt.
Markdown and AsciiDoc syntax are extremely [0]similar, and AsciiDoc supports markdown's syntax.
IBM, Microsoft and Stripe all have tooling that enables Markdown includes/partials. None of those will render natively in GitHub or Gitlab - but if you really need that feature you probably don't care as the AsciiDoc support is also so poor.
I think if AsciiDoc and Markdown were in the same league from a writing and reading perspective, AsciiDoc might have more traction. Other comments in this thread support the case that it's just harder to write and read as markup.
A whole other topic is how Markdown has been adopted by React-driven systems. This just solidifies Markdown's position as the lingua franca.
But it is probably just plainly "it happened that Big Things adopted it, so more Big Things saw it and also adopted it"
I'm currently doing an architecture decision record about Markdown documentation, and will add Markdoc to the candidates. The leaders so far are MDX 2 with plugins for JSX-style work, and Svelte for a fully dynamic site.
I'm aware of the Markdoc page about "Why not MDX?" which explains that Markdoc is deliberately less capable than MDX. But the page doesn't show how to do typical needs (IMHO) such as loops or substitutions. And for simple writing, compare with standard markdown annotated with tags/templates using Liquid or Jinja or similar?
I haven't evaluated Markdoc fully, but I did have some experience using both MDX 2 and markdown-it—the parser used by Markdoc. I can honestly say that MDX 2 (and remark for that matter) is a lot slower in parsing bigger markdown files. Anecdotally, I saw markdown-it outperforming remark by a factor of 20 in terms of generating the file's AST alone.
Markdoc with its added complexity may add some penalty to the parsing performance, but does it do so to a point where it gets considerably closer to MDX? I doubt that this is the case, but would love to see someone verify this.
The primary difference between MDX and Markdoc is that an MDX article is essentially imperative code (like a typical React function containing JSX), whereas a Markdoc article is purely declarative. That makes it easier to reason about what's being rendered, and makes things like static analysis much more straightforward.
With MDX, you can essentially write arbitrary JavaScript code in articles. We had a similar situation when we used ERB, and it resulted in markup which was easy to write but very difficult to read. Markdoc strikes a better balance between the two in part because it has better guardrails and stronger division between content and code. It is a tradeoff, though.
If I’m honest though, the one thing that can beat it for me is my guilty pleasure - pug in Svelte. If a better way to author web pages exists, I’ve never seen it!
Special shout out to:
@marcioAlmada for providing us with the @markdoc GitHub org.
@koomen for gifting us https://markdoc.dev.Documentation is information intended for humans to understand complex topics. Code is information intended for machines to understand complex topics. Code makes shitty documentation. Code is time-consuming, error-prone, nit-picky, and extremely verbose compared to the kinds of tools designed for humans: WISYWIG editors, buttons, windows, graphics, diagrams, audio, video.
Developers today have grown up in a world where doing anything requires writing code. They literally do not understand how to solve problems without writing code. And have even fooled themselves into thinking that writing code is a superior way for humans to solve a problem than clicking a button. So rather than have robust tools that solve our problems, we now only make tools that require us to create our own customized tools to solve our problems. Forever making slapped-together jigs for every project, rather than just buying and using a premade tool.
Confluence is what a documentation-creating solution should be. Not only does it have a WYSIWYG editor, it has custom plugins, rich content, and dynamic components, all of which can be added anywhere, immediately, with a live preview, with no training, no code, no syntax. Simply an interface for an average human to make a beautiful, useful document, quickly and easily.
From their page: "I like Markdoc because it lets us still do anything we want with code in the docs without bogging down the content authoring experience. If we need some new component, designers and engineers can whip that up. So as a writer, I can work in the docs content and stay focused."
You can do the same thing with Confluence. They just had Not-Invented-Here syndrome, and wanted to do something with their excess engineers.
Your central thesis is that code and documentation are two very different things. Donald Knuth, who knows a thing or two about writing both documentation and code, invented literate programming because he disagrees with that. The key point of literate programming is that writing good code is a form of documenting your thought process for the benefit of other programmers who may have to maintain and understand your code.
Confluence is part of the reason I run an Atlassian free company. I've never been impressed with Atlassian tooling. Most of the tools I use instead support things like Markdown and some other sane things. It's fine as a poor man's word processor but it's a bit of a straight jacket. Don't get me started on Jira.
Am I aware that Confluence is a product created by a company for a profit in order to provide an advanced implementation of a design for specific use cases? Yes.
> And it's not even a particularly good one.
I have used 20 different wikis. None of them provide all Confluence's features or are nearly as user-friendly. Very few of them provide a robust API. Very few of them have as seamless a WYSIWYG, not to mention native Markdown support. Very few of them have advanced management capabilities. None of them have as many advanced dynamic content features, very few of them have marketplaces. It is hands-down the best tool for what it is designed for.
> Donald Knuth, who knows a thing or two about writing both documentation and code, invented literate programming because he disagrees with that.
Literate programming is ridiculous. The idea is to use a human language to tell a story and have that story be converted into computer code, with the idea that the human will understand the story better and somehow this will result in them being able to write better code. But that's stupid for two reasons: 1) writing a story is hard. communication is hard. most people just suck at it. the idea that a human will be automatically good at telling a story is in itself ridiculous. 2) humans just suck at writing code, no matter how well they understand its purpose. Traditional architecture is intended to formalize the purpose and function of the program and can result in perfectly adequate code, assuming the human actually learns how to do their job properly, which I admit is a pretty big ask for most people in tech today.
If you don't like Confluence and Jira it's probably because you haven't learned how to use them (or you just don't like that they're not free). They are incredibly flexible and extensible and the marketplace is full of tools to expand their functionality, even though the base functionality is already very powerful.
Confluence UX is horrible. There are a number of reasons for that, but just one is enough for me to reject it: you can't create or edit content in your own, personally optimized, automatable, text editing IDE. (And if you actually can, guess what? That's another argument against the usability of confluence, because I was either unable to find or unable to configure such a feature)
Obviously it's important for an average user to be able to edit content. But there are options that don't lock out the programmers who legitimately prefer plaintext editing.
They are making documentation for developers.
And in fact, having good documentation and developer tooling is a core part of their value proposition.
Making it easy to look at documentation and immediately implement something that works is a core part of the Stripe UX.
So documentation can be considered as part of their product offering.
In that way it makes sense for them for it to be driven by code.
For what it's worth, we considered MDX, but chose not to use it. Full explanation here: https://markdoc.dev/docs/faq#why-not-mdx
Doesn't this syntax:
<callout type="check">...</callout>
{% callout type="check" %}...{% /callout %}
?I think that perhaps there is also a contingency of people that have been using templates for ages that have the "{% %}" style, so this is maybe attractive to those folks?
To be clear, I agree with you, but I am just trying to figure out why this syntax would be chosen.
Edit: Perhaps there is also some sort of HTML injection argument against using real tags? That is to say, if you have an interface that allows users to input markdown, then with "{% %}" you can easily filter the allowable template tags, but perhaps it is just more error prone to try to handle "<>" tags that might then themselves get inserted into live HTML. I haven't thought it all the way through but just wondering if there is a non-stylistic argument for it.
Basically because MDX mixes JSX and markdown, you need knowledge of JSX/JS (which non-devs might not have), and tooling dedicated to build, parse it and so on. Markdoc is more of a "separation of concerns" approach.
If all you're producing is HTML - which is 99% of what Markdown is used for - having to deal with a raw AST in order to modify a document is like deciding to use Assembly instead of Python. Sure, it could be more efficient if you really want to spend the time, but it's generally a step backwards in every way.
I've personally decided that the only predictable, reliable and maintainable way to deal with Markdown is to extract the frontmatter, then convert the rest into bog-standard CommonMark HTML and then use JSDOM to do any additional manipulation. So instead of fighting with some wonky AST tree and APIs, I can use the DOM and standard web tools and code.
Markdown parsers will let you pass through HTML tags. You can define any tag you want. There is no difference between:
{% callout type="check" %}
{% /callout %}
<callout type="check">
</callout>
<xsl:if test="price = 10">
</xsl:if>
Honestly, Markdown needs to be killed with prejudice. So much wasted time and effort getting it to do what people need it to do.
It seems to work pretty well as a human-editable format for rich text. Is there a different format you'd like to see take its place for that niche?
What I would like to see is a new self-contained Web Document standard (none of the various implementations out there qualify) that mimics the core reason Markdown and other plain-text systems like AsciiDoc or LaTeX exist: To separate the writing from the presentation, but with some basic formatting as needed for most documents.
There are various self-contained document formats out there: ePub and mobi files use HTML inside, as does Microsoft's CHM. And there's a hundred zipped XML file formats out there - docx, odt, etc. But they're either write-only, proprietary or are too complicated for this purpose.
What I would want is a simple .wdoc standard file, which is either a plain-text or zip file containing a very strict subset of HTML and CSS which basically mimics the output of Markdown. (It could be called MarkUp, actually). The subset would be limited to just semantic tags and reasonable formatting, to guarantee editable HTML. Nothing dynamic or crazy. Just pure WYSIWYG.
If it was a W3C standard, there could even be a new HTML tag <doc><\doc> which wraps raw .wdoc markup in a sandbox, guaranteeing that nothing inside those wrapper tags will display anything but the allowed styles and tags. A zipped .wdoc (with images) could be included with a src attribute: <doc src="...">, with an "editable" attribute that defaults to false, but could be flipped to allow editing. Maybe an "allowed" attribute to limit formatting even further.
Like the video and audio tags, basic editor features could be supported natively, but would also allow custom editor skins like CKEditor, TinyMCE, Trix etc. But again, with standard output. This would be great for online forums like HN or reddit. In standalone apps, like Apple's Text Editor or Microsoft's WordPad, the output would be a cross platform rich text document that is readable and writable by any browser or standard .wdoc editor.
The idea is to Keep It Simple Stupid, but also provide basic cross-platform WYSIWYG editing where the simple, clean formatting is always displayed exactly like it looks when editing. I use Typora, which is a great little rich text editor that uses WebKit for the interface, and then exports Markdown, which I then process into a web page. It's insane. Let's cut out the useless middle step.
Browser engines have progressed so far since Markdown was created. It's all a matter of standardization at this point. Keep the spec simple and focused on just creating simple documents. If someone wants to use the output as a full-on web page, then it's just a matter of not using the <doc> wrapper and adding full-strength CSS, JavaScript, etc. The CommonMark spec could even be updated so that .wdoc is the standard output of a processed .md text file.
The web has tilted too far towards the dynamic app end of the spectrum, and lost its roots as a document format. I think something like this would be a great way to get back to that.
Personally I've always preferred HTML tags to markdown, primarily because I can usually understand simple HTML by looking at it, while the markdown needs a cheatsheet if I haven't been working on it recently.
I think this could probably be done as a web component if the browsers didn't implement it directly.
Plus most GUI tools produce HTML and that could probably be modified to output a more restricted format.
For example:
```bash
curl https://api.example.com/stuff \
-u {% apiKey /%}: \
-d "foo=bar"
```
```bash
curl https://api.example.com/stuff \
-u MY_API_KEY: \
-d "foo=bar"
```
If you render the site to static HTML, though, you will have to do something more like a last-minute search-and-replace. Instead of rendering raw HTML, you could render the entire site to either the Markdoc AST or Markdoc renderable tree, which are both serializable. That's the approach that I use on my personal site [0], which implements a system similar to ContentLayer. [1]
[0] https://github.com/nkohari/nate.io/blob/master/build/Content...
I wondered how this works and if it's FOSS.
This looks so similar to that.
The key advantage of using Markdoc instead of using Markdown with liquid or another string-based templating system that preprocess the content is that the tags and other custom syntax are a first-class part of the Markdoc format. The document parses to an AST and the individual tags can programmatically manipulate the content and document node hierarchy instead of just manipulating or outputting strings of text that are passed on to a Markdown processor.
Perhaps an example may be useful? Or do you have an example where Markdoc custom tags are especially different/better than Liquid custom tags?
Markdoc custom tag syntax:
{% bold %}
Lorem ipsum
{% /bold %}
{% bold %}
Lorem ipsum
{% endbold %}
export const bold = {
render: 'Bold'
…
};
import \* as React from 'react';
function Foo({ children }) {
return (
<b>{children}</b>
);
}
return Markdoc.renderers.react(
content, React, {
components: {
Bold: Bold
}
}
);
class Bold < Liquid::Block
def initialize(_tag_name, _content, parse_context)
super
# Do whatever you want here,
# such as parsing to your favorite AST,
# or calling a DB, or RPC, or API, etc.
end
def render(context)
# Do whatever you want here,
# such as processing your favorite AST,
# or using the rend context vars, etc.
"<b>" + super.strip + "</b>"
end
end
Liquid::Template.register_tag("bold", Bold)
print Liquid::Template.parse(File.read("my.txt")).render