Plugins case study: mdBook preprocessors | Hacker News Reader

Plugins case study: mdBook preprocessors

eli.thegreenplace.net

23 pointschmaynard14 comments

Biggest problem of mdBook is the lack of the good PDF and ePub export[1]. This is why Quarto[2] (based on pandoc) is a better choice if you need both online and offline rendered documentation/text. And Typst (or LaTeX for more conservative folks) for the offline documentation/articles/books.

[1] https://github.com/rust-lang/mdBook/issues/815

[2] https://quarto.org/

I get where you're coming from but I'm not sure if PDF generation was a goal for mdbook to begin with. I'm pretty sure they're working on it now, but I'd stick to a domain specific tool, personally I find typst excellent to generate documents for prints.

For simple HTML docs mdbook is also excellent. I don't know if you could combine these two domains into one tool nicely. To me they're just too different.

although I have made presentations with mdbook using custom written preprocessors and a custom renderer (all of which were extremely crude, but did the job)

Quarto handles both beautifully, just look at their gallery[1].

[1] https://quarto.org/docs/gallery/

I already think that markdown is barely ok for writing documentation, and the experience of plugins in mdBook is why I tell people not to use it (edit: it = mdBook). The base flavor of mdBook is minimalistic. Maybe that’s a good thing, that you’re given a minimalistic markdown as a starting point? But if it’s minimalistic, then it’s certainly missing some things that I’d want to use in the documentation I write, and the experience of using plugins is, well, not very good.

My current recommendations are MkDocs (material theme), Jekyll, and Docusaurus. Hugo gets a qualified recommendation, and I only recommend mdBook for people who are doing Rust stuff.

What is missing from markdown? mdbook also uses in some parts the GH flavored one, so you can create notes [1] and similar. On top of that, you can add support for Mermaid.

Personally, I don't think you need more than that for 90% of the documentation, but I'm happy to hear more about your use case.

[1]: https://github.com/orgs/community/discussions/16925

Markdown is devalued as a format because of the bizarre shortage of Markdown VIEWERS. You find Markdown documents in every open-source project, and you always wind up viewing them with all the embedded formatting characters. Why?

Why provide documentation in a format that is so poorly supported by READERS? Or, to respect the chicken-&-egg problem here: Why is there such a shortage of Markdown viewers?

Every time this comes up, respondents always cite this or that EDITOR that has Markdown "preview." NO. We're not editing the documentation; we're just reading it. So why do we have to load the document into an editor and then invoke a "preview" of it? Consider how nonsensical the term "preview" is in that case: What are we "previewing" its appearance in, given the dearth of Markdown readers?

If you're looking for a CLI markdown reader then I'd recommend glow.

https://github.com/charmbracelet/glow

Thanks. But that's even stranger! I just want to double-click a Markdown file in my OS's file browser and read it, with formatting applied.

Ahh, if you're on KDE then Okular should do the trick, it should even update live when you update the markdown file.

Thanks!

Can you speak on what features were missing ootb and which plugins respectively gave you trouble? I'm not sure how "people who are doing Rust stuff" would specifically get more out of it either? Are you implying you cant just use the tool/plugins without familiarity in Rust? This is not my experience.

I have maintained a pandoc filter in Haskell for a while. Pandoc is written in Haskell, and so takes a similar approach: JSON API for arbitrary languages, but with a library for trivially parsing that JSON back into the same Haskell datatype that Pandoc uses under the hood.

When you sit down to write the filter for the first time, it’s amazing. You’re using a typed IR that’s well documented, a language that catches you when you’re making mistakes, etc. You have to do very little boring grunt work and focus only on what the filter needs to do.

Over time, the filter became feature complete, so I didn’t want to have to touch it anymore, but the library for parsing JSON releases a new version for every new feature in the AST, and the parsing function checks that the version your filter was compiled with is at least as new as the pandoc that produced the JSON. It has to, because if the pandoc is newer, your older filter won’t know how to parse some of the nodes.

My filter is feature complete, and shouldn’t need to look at those nodes or their new fields: needing to upgrade is just toil. But over time, pandoc releases new versions, and I’d need to recompile the filter to build the new version. At those points, I also found myself having to deal with library, build tool, OS, or compiler upgrades, all to recompile a filter that should need to be.

Eventually I switched to Pandoc lua filters, which eliminated the toil while also being platform agnostic (and not requiring any sort of notarization or executable quarantine on enterprise systems) at the expense of having to tolerate Lua the language. Now, new versions of Pandoc don’t require me to boop a version number in my filter. If that wasn’t an option, for any future filters I write, I’d write my own JSON parser that only parses as much of the JSON as I needed, leaving the rest untouched—that way it wouldn’t matter if new changes came along. I could even tolerate backwards incompatible changes as long as they didn’t alter the contract of the narrow focus of that one filter!

There are of course other ways to deal with problems like these (protocol buffers, JSON parsing with an option to throw away unrecognized JSON, etc. etc.)

I have not looked at how mdBook plugins handle this. But if I were writing such a plugin, it’s the first thing I’d look at, and be sure to program around.

Mdbook passes a version and the renderer/preprocessor can/should do a version check. Since it uses semantic versioning I would expect it to abide by those rules.

There's some example code in their docs (fn handle_preprocessing in the no-op preprocessor) which I've actually included in a preprocessor I wrote some time ago. https://rust-lang.github.io/mdBook/for_developers/preprocess...

You mean plug-ins?

Reply on news.ycombinator.com