Docbook
docbook.org
docbook.org
Today I'm using AsciiDoc with AsciiDoctor. The AsciiDoc syntax is as pleasant as MarkDown, but much more powerful and more consistent. AsciiDoc can be converted to DocBook if needed, but I've never needed this possibility, because all I need is PDF and HTML output.
https://oreillymedia.github.io/HTMLBook/
As far as I know, O'Reilly Media still uses this for all book production.
Neither, AsciiDoc has similar design goals to Markdown but it's a completely separate markup language. I like to explain AsciiDoc as a markup language that sits somewhere between Markdown and LaTeX, it handles far more things than Markdown (and is therefore more complex) but isn't nearly as complex as LaTeX (but also doesn't handle nearly as many things).
Behind the scenes, you're actually getting well-formed HTML5—specifically, HTMLBook, our new open standard for writing books (or book-like projects) in HTML.
If you prefer writing in one of the lightweight markup languages popular today, then we've got you covered with the Code Editor. In this text markup mode, you can use Markdown, DocBook XML, or AsciiDoc, a language similar to Markdown but more robust for presenting complex information.
20 years ago, there was Docbook, an SGML application. There was also Linuxdoc SGML, another, simpler SGML application for us common people. You had to write the SGML in an ordinary text editor, but the markup isn't too obtrusive. Stylesheets were written in DSSSL (essentially Scheme) and processed with Jade. Output was either HTML or flow objects, which could be processed by TeX to get nice postscript. There were also some shortcuts to produce Unix man pages.
And then the whole world apparently decided that SGML is bad and XML is good, and started over. Markup became annoying, DSSSL was replaced with XSLT, which should not be looked at with unprotected eyes, instead of Jade we got Apache Xalan(?), which needed a supercomputer to run (thank you, Java) and was generally a pain to use (again, thank you, Java), and it produced either HTML or XSL:FO, and there was no working processor for the latter, so there was no longer a good way to produce PS or PDF.
It took a few years until xsltproc finally made XSLT usable (but still not pretty to look at), and Apache FOP could format to PDF (when running on a supercomputer, just like Xalan). But nobody wrote their documentation in Docbook or Linuxdoc anymore. Because markdown and asciidoc and a bazillion similar ad-hoc languages may not have been as theoretically clean, but at least the tools worked, even on a cheap workstation.
I’m prone to Stockholm syndrome relationships with technology, but XML Docbook / XSLT eventually popped my self delusion bubble.
I wrote a perl script which converted their docbook into html by applying XSLT to it. Worked pretty well. They just run a script to generate the new version of the website each time they do a release.
Apache FOP / Batik is okay for generating nice looking PDFs, but its SVG rasterization is much worse than that of Chrome's PDF renderer (Skia).
As I didn't get the memo about XML being dead, I'm still using Docbook and XSLT/XInclude to structure and filter content written in markdown/rst.
But I will be switching over from XSL-FO to CSS paged media for layout as soon as Chrome supports bookmarks in PDF output, or either WeasyPrint or wkhtmltopdf improve their CSS3 column support.
Should I just go for Tex once I get the irresistible itch to do this sort of stuff? What's the web-first solution that would still permit to also support the godsend Epub and the pathetic PDF? I'm also up for sacrificing some of my life on the altar of Lisp―is there anything like Butterick's ‘Pollen’ with any popularity?
There’s a few high cost options used by technical writers that are likely overkill for what you need (Madcap Flare is one I relatively enjoy).
Sphinx is probably the best for what you’re looking for though: http://www.sphinx-doc.org
HTML does not have the fidelity of TeX. You just can't do the same things in HTML that you can do in TeX. Just to make it clear, TeX is a real programming language, though with a truly bizarre syntax and grammar (I still love it, though ;-) ). Trying to execute TeX and output HTML is incredibly difficult to do with any kind of quality. You could take LaTeX documents and then write a completely different formatting system that output HTML, but at that point, you would be better off using a system that is intended to output HTML: like docbook :-)
I dream of the day when PDF is as dead as Flash, for I have dozens of PDFs in my reading pile—which I can't reasonably get to, because I don't have a 14" tablet or a printer and can't spend days on end propped in front of the laptop just to read through them. Meanwhile, practically all of those docs are two-column walls of text alternating with rectangular pictures, nothing more complicated than Markdown, and they won't be printed in a paper journal. But the authors felt for some reason that they have to employ PDF.
I've been sermonizing for years that it'd be better to publish in HTML and, if the author feels like it, build PDF from the same source—and now it turns out that Tex is a one-track thing and I could just as well ‘print’ HTML to PDF! I gotta see what's so special about Tex that HTML doesn't look serious enough to the big boys.
Like I said, TeX is a typesetting language. It is literally a programming language that takes typesetting programs (including the data) and outputs an exact photo-ready output. It's very easy to convert the photo-ready output to a PDF, or a PNG, or GIF, SVG, etc. Because TeX is a programming language, you can write macros (there are no functions in TeX). These macros work on input you give it. In the old, old days, someone would create a set of macros for typesetting documents for each of the major journals and publications you might want to publish to. Eventually, a guy name Leslie Lamport wrote a whole set of really good macros called LaTeX that would allow you to write documents in a consistent way for a whole bunch of different types of publications. That way you could minimally change the document and resubmit it to a different publisher.
So why do people still you TeX? There are two main reasons: First because there is still a need for very high quality print output. TeX is special in that it has perfect algorithms for doing things like line breaking, kerning, etc, etc, etc. The output of TeX is dramatically better than just about any other piece of software. It's insanely better than any word processor will output and there are probably only a handful of desktop publishing packages that are even close. If you need absolutely perfect print output, then TeX (and LaTeX) are a really good fit.
The other reason for using LaTeX (not so much TeX, because typesetting documents in TeX is almost insane: I used to do it for fun, but I have never, ever in my career met another person who has done this) is for typesetting mathematics. The LaTeX macros for math are extremely convenient and the pictorial output from TeX are perfect. This allows you to typeset your mathematics in LaTeX, generate a picture and then paste that picture into an HTML document, or Word document, or whatever.
There is one notable file format that is built on top of TeX that also is meant for digital use: Texinfo. This is the original documentation format for GNU. You can write hyperlink documents in Texinfo and it will generate TeX output for high quality print documents, and HTML (as well as a few other formats). It is very limited in its features, though and the HTML output is so-so, IMHO.
There are a couple of other document formatting systems that output TeX as well (and in fact, I'm pretty sure there is one that will allow you to build some rudimentary TeX documents with docbook), but I'll let you look for those if you are interested.
Because Donald Knuth.
Note that Texinfo was originally meant to generate output for the info file format (viewed by the GNU info viewer as well as GNU emacs and a few other less used viewers) alongside TeX, which is probably where it got its name (tex+info output). The info format is essentially a primitive hypertext format with support for hierarchical nodes and keyword indices packaged in a single file (essentially a primitive text-only form of CHM, though later versions of info also support external image references). The HTML and other output formats were added much later.
I asked a friend, and apparently some journals are still printed on paper but no one buys them on paper. So the practical reason for the use of PDFs seems to be that people can print them at home to avoid reading from the screen.
One actual reason for non-adoption of HTML, that I can see, is that there's lots of tooling available for collecting, organizing and annotating PDFs. But it doesn't look insurmountable, since files with metadata could just as well be HTML-based, and there are already multiple tools exactly for annotating HTML, only they position themselves for different audiences.
The thing is ePubs and such aren't all that easy to consume. They don't provide standardised predictable layout, so you can't control how it will be presented to the reader. HTML and ePub only provide layout hints to the device or reader app. Even the reader doesn't have much control how the document is displayed. In contrast PDF provides complete, precise control over presentation. If it looks bad or is misleading due to layout issues or truncated tables and such, all of which are possible and can be very significant in scientific or technical material, it's all on the publisher.
I read a ton of PDFs, but then I'm a gamer and most tabletop RPGs are published in PDFs. In that case it's because many of them contain a lot of tables, forms and illustrations. They're also trying to create an atmosphere and express a theme, so layout matters. For a lot of material layout makes such a big difference to how material is interpreted that it is content.
Meanwhile, I'm browsing the web all day every day and yet to see what is there to fail in a simple text-and-images, black on white layout that most academical and technical papers need. Complex formulas and legend positioning is solved with images for now—and I'd guess that with some industry effort MathML and SVG could replace them. Discrepancy in the support of HTML formatting features is solved with standardization of the current state-of-the-art for some ‘paper viewing software,’ just as PDF is standardized. And regular browsers could switch on the ‘paper reader’ mode for the format.
IMO it's inevitable that fixed sizes for layout will go away, for the simple reason that it's not how screens work. The sooner it happens in tech writing, the happier the rest of my life will be.
Also I just remembered that one studio specifically makes online ‘books’ to control the layout on the screen: https://bureau.rocks/books/ (though the use of the scroll-activation is questionable to me). Of course, they likely had to expend considerably more effort than is needed for either regular pages or PDFs. Alas, their explanation of the reasoning is not in English—basically, they don't like how formats like Epub are displayed but don't even consider PDF: https://bureau.ru/books/manifesto/
Ok, failure modes. HTML does not have the concept of a deterministic page size and length or predictable pagination. In academic papers references, figures, footnotes and titles need to be presented in a deterministic way so that when text refers to a figure, or a footnote is referenced, etc, they exist in a predictable and know relationship to each other. If the text refers to a figure that is supposed to be on the same page, the author wants to be absolutely sure it appears on that page. HTML can't even guarantee that a given image will be displayed at all. have you never been to a web page where some images were not shown or just appeared as placeholder boxes? Device and image library and support limitations might lead to an image not appearing at all, or at an unpredictable relative size so it's not easily readable, appearing in a form such that details are not easily discernible such as with bad gamma or transparency settings, or with colours not shown with fidelity. This can actually change the meaning or interpretation of a result.
Information that belongs together, that the author intends and needs to be together, had better be together. None of this can be guaranteed by HTML or ebook formats. These formats can't guarantee what fonts information will be represented in, they can't even guarantee whether text will he shown in italics, bold, what relative font sizes they will have, etc. Reference formats are very precisely specified, but these formats simply don't support that level of precision.
So ok, you can say you don't care about these levels of precision, but your asking why other people do and these are the reasons.
Solved with references to the structure, e.g. ‘Figure 1.1‘ or ‘Section 2.4.‘
> If the text refers to a figure that is supposed to be on the same page, the author wants to be absolutely sure it appears on that page.
Why? Place the figure right after the text, or before it. “Supposed to be on the same page” is an example of thinking in terms of layout inherited from printed media. Delivering information doesn't depend on there being a concept of pages.
> have you never been to a web page where some images were not shown or just appeared as placeholder boxes?
Solved with a format that ships the page and images together, like Epub does. It's a basic requirement for sane personal storage-for-reference anyway.
> Device and image library and support limitations might lead to an image not appearing at all
Image support is solved with standardization of the format, as are many other formatting support features. If one wants to read docs on something less capable, like low-end devices, they take a conscious trade-off for the risk of something being unsupported.
> or at an unpredictable relative size so it's not easily readable
Don't quite see what kind of issues you mean here. Size relatively to the text is rather easily specified, and afaik normally honored unless the image is of some wild proportions like extra-wide.
But actually, I really wish that browsers and readers could zoom in/out on any image. I already do this frequently with Mac's ‘zoom’ tool. It's probably not even too hard to implement, I should look into finding or making a browser extension for this.
> such as with bad gamma or transparency settings, or with colours not shown with fidelity. This can actually change the meaning or interpretation of a result.
Interesting point, however I'm hard-pressed to see practical instances of this. When academic PDFs are explicitly made to be printed, apparently usually on office or home printers, I doubt that high quality of the result is to be expected. In this situation, authors should probably opt for rugged but dependable looks in most cases, with simple gray-color graphics and easily distinguished shapes, since even color print is not of much certainty. If precise color graphics are necessary then the paper is not fully readable in the printed form favored by many people, and I guess they will have to refer to the images on their computer—though even then I doubt many of them calibrate their monitors. So, I don't see much difference with HTML being read on a phone or a tablet an ebook reader, especially since phones and tablets have pretty good screens and support for HTML features. Finally, for strict reference, precise graphics can be delivered as attachments in a stricter format—just like it's done nowadays for large color visualizations, afaik.
> Information that belongs together, that the author intends and needs to be together, had better be together.
Again, put images after text that pertains to them.
> These formats can't guarantee what fonts information will be represented in
Why is that necessary? For math symbols, Unicode has several variations like Fraktur.
> they can't even guarantee whether text will he shown in italics, bold
Sure they can.
> what relative font sizes they will have
Subtext and supertext are respectively smaller and larger than the normal type. What else is needed? Do people write “let polar coordinates be denoted with type of 36 points”?
Donald Knuth decided that there is value in having a system that for all time will produce exactly the output the author intended 20 years or 200 years later.
Every TeX system will layout a document in exactly the same way within the limits of the output device—internally TeX computes positions with fixed point arithmetic at units smaller than light wavelengths.
There is a tremendous ecosystem that runs on top of TeX written in the TeX macro based language. This includes the popular LaTeX system, hundreds of packages including tools for example to typeset musical scores, chess articles, and everything else you can imagine. Do note that some of these packages are very high quality—many comercial books in difficult to typeset subject like Physics, Mathematics, and CS are written using TeX.
My company is now moving to putting their docs in confluence, which I hate.
The only good thing I've found so far about Confluence is that you can kind-of-sort-of escape its WYSIWYGness by pressing the small “<>” (“Edit Source”) button at the top and just paste your XHTML, that you've written like a normal human being, in there. But of course it only supports a subset of XHTML, doesn't support anything CSS beyond very basic styling, and it makes it very hard to work with tables of contents and other extensions.
Still better than SharePoint! ... Though I've taken to writing as much in markdown and storing with my code as possible. Our build process can compile code in the docs to ensure that it stays in sync