Ask HN: Best tools for self-authoring books in 2023? | Hacker News Reader

Ask HN: Best tools for self-authoring books in 2023?

68 pointsfactorialboy57 comments

Hey folks,

I'm a fairly "nerdy" person who loves to spend time in text-editors, terminals and (if needed) browsers .. and I'm trying to understand the self-authoring tooling landscape suitable for me.

My work will not contain mathematical formulae, nor would it contain code snippets. But I would like to include images, or generate diagrams (flow charts, event sequence diagrams etc.)

A quick Google shows me mkbook and mkdocs. Both of these take in markdown files and generate HTMLs. I plan to evaluate these.

Any other authoring frameworks worth considering?

What are my options in terms of self-publishing? Amazon still king? What other stage of self-authoring would I need tooling for?

Ah, must mention I primarily use Fedora but I'd like to sync the repo on my MBP as well, and would like to work seamlessly on both machines.

Any direction would be helpful. Ty.

View on HN

Tooling is mostly unimportant. Writing the actual book should consume much more of your time. Unless you have very specific requirements around the output (e.g. one of my books has animations so I wanted something that made them easy to embed: https://www.creativescala.org/creative-scala/cycles/interpol...) spending time on tooling is yak shaving. Choose something using a tool chain you can live with and move on.

For selling books, try to do most of your sales via your own site. Gumroad is a good partner for selling PDFs. Lulu is pretty good for printed books. Amazon is a necessary evil due to their reach, but they will charge a much higher fee than doing it yourself and they are evil. If you don't self publish your royalty rate will be such that except for the few best selling books you won't make any significant money. It can be worth going with an established publisher if you want the prestige (and perhaps you can leverage that into $s) but in most cases I think self publishing is a better option.

Let me again emphasize writing the actual book is much more important than messing around with tools.

This 100%. Make sure you spend no more than 20% of you time in tools. For most people 10% is better,but since you are doing math plain text isn't good and so I give you a little more time. But always focus on content first.

In addition to concentrating on the writing, just keep the content simple with minimal markup if possible so that it's easy to rejig later if needs be.

People rag on Amazon, but as a book buyer it's nice to have all your books in one place on a platform that's likely to be around next year, rather than random PDF's that end up who knows where. It's also nice that the book is subjected to reviews. Those can be capricious and unfair at times, but if there are enough, you get a better sense of the book than "Buy it! Trust me!".

KeenWrite is my free, cross-platform desktop Markdown editor, which leverages Kroki to create diagrams based on textual descriptions. I developed the software to help write a sci-fi novel. The novel has speech bubbles, simultaneous conversations, raster images, vector illustrations, R plots, and fancy chapters. I wrote KeenWrite because I wanted a way to reference a character sheet from within the document by way of variables.

KeenWrite is compatible with pandoc and knitr. In effect, it's a replacement for the shell scripts I wrote about in my Typesetting Markdown series. The series describes using pandoc and ConTeXt to typeset Markdown documents.

Typesetting to PDF is accomplished using ConTeXt, which is a typesetting system with a strong focus on separating content from presentation.

* https://dave.autonoma.ca/blog/2019/05/22/typesetting-markdow...

* https://keenwrite.com/screenshots.html

* https://keenwrite.com/

* https://kroki.io/

This looks excellent! Thank you for building it.

I hope you release it in EPUB, which is a popular format for digital books. Like MP3 for books.

I know the following will get downvoted, but I hope you upload it to Z-Library too. Z-Library is a website that allows people to illegally download copyrighted books for free. If your book has even a tiny bit of success, it will end up there anyway.

I second EPUB. Works perfectly with Kindle-copycats, which do not have DRM issues. Otherwise, I'd recommend LaTeX given your requirements.

Write first, layout second.

Layouting is an entirely different job than writing. For long and very long textual content, I have yet to find automated results that give me control and look professional [0].

If you want to stay FOSS, learn Scribus. Becoming decent at typography and layouting is a bit of a journey, but I found it to be not too bad. If spending some money is okay, I'd go for Affinity Publisher.

None of those tools can do ePub. ePub is a format I've grown to hate; my pipeline currently consists of custom python -> pandoc -> custom python -> manual fixing.

For writing, don't worry too much. Use a program that you're familiar with and know how to use. If you don't write fiction with highly complex custom worldbuilding, a normal word processor is probably fine. I personally write urban fantasy novels, and I use a combination of Word and OneNote to manage everything as long as I am in the writing phase. It's more important to be easily able to change the layout than to have it finalized. For example, I like proof reading on Normseiten (A4, 30 Lines per page, less than 60 characters per line, monospaced font - I use Comic Mono) which is distinct from my usual writing format (Adobe Garamond, 12cm by 19cm pages with uniform 1.5cm borders).

> What are my options in terms of self-publishing?

Depends on the subject matter. If you're writing is any good, might as well try and score a distribution deal (at which point most of the layouting things will be out of your hands anyway).

All the best for your book project.

[0]: LaTeX sure looks nice, but gives me no control at all.

Honestly, whatever suite of tools allows you to organize your research, thoughts, and easily write text. Remove as much friction as possible. When we wrote The Workshop Survival Guide (https://www.workshopsurvival.com/), we used post-it notes on a wall and google docs. It allowed for a fluid collaboration.

On a more meta note, how you approach writing your book is probably more impactful than your toolchain. If you're working on a book that educates (rather than, say, entertains), I'd like to share the writing framework that my longtime collaborator, robfitz, describes in Write Useful Books (https://www.usefulbooks.com/). Basically, treat your book like a product, find your audience, and test your subject matter with them as your craft your prose.

* Disclaimer: Rob and I run Useful Books, a community and toolset for non-fiction authors.

You might be interested in Pollen: https://docs.racket-lang.org/pollen/

Imagine a markup language built by a typographer who also is a pretty darn good programmer.

Seconded, Pollen is outstanding if you only aim at online books.

Much depends on the topics you cover and the target readers you want to reach. As others have noted, ePub is the standard for ebook publishing. But if you want to publish a printed edition as well in the future, your toolchain would have to support pdf as well.

Pandoc can convert almost everything in everything and is great for terminal nerds. However, be aware that formatting pdfs for common self publishing distributors like KDP (Amazon) has some strict requirements.

For someone without the nerd dns, Vellum is a great software on the Mac that formats ePub and pdf as required for different vendors and print formats. The best feature is that you have one manuscript and export to ePub and pdf. Including graphics works quite well, but sometimes clever positioning for pdf is a bit tricky. Be aware that Vellum is a paid software and it is not suited for equations and docs that require very formal formatting (on the other hand, ePub ebooks are generally not a very good choice here).

But before you choose your toolchain, be aware that writing and formatting the manuscript is only a small part of self publishing. You need distribution and marketing, proof-reading and a business model.

Amazon can help with distribution, but you could publish via your own website, Gumroad and numerous other platforms. Do you want to only sell the ePub? Or sell add-ons, courses, bundles etc.? Amazon is okay for mass market topics, however there is a large competition and you have to have a marketing strategy. Selling via Gumroad allows for bundles and greater flexibility, but is better suited for niche topics. Again: You have to do the marketing.

Does your target group prefer printed books? Amazon/KDP can help here with print on demand. Beware anything where you print (and buy) hundreds or thousands of books in advance: You will have your garage filled with them and never sell them all because of logistics.

So, please don't only focus on the toolchain and the doc to ebook step. That is one of the easier parts in the equation.

My wife and I are writing with Scrivener (she on OSX, me on W11), I use Leanpub/MD to publish. We used Affinity Publisher to create a version for print and Affinity Designer for the cover and all artwork.

Leanpub is easy and works.

I would consider Obsidian for my next venture as Scrivener has too many features for me that I don't need.

For my poem book I wrote some code that generated mazes for the cover [0]

We do go with Amazon for ebooks but not for printing, their quality is way below what we find acceptable (we're in Germany and found a printer in Poland after being let down by our German printer).

[0] https://www.amazon.de/Olavsweg-Gedichte-Gedanken-lyrische-In...

I've used Scrivener off and on over the years, typically for the earlier stages of getting a book organized and material pulled together. But at some point--and especially if I'm collaborating at all--I end up pulling things into a word processor to start crafting things into a final form.

Scrivener was, I believe, originally designed for screenplays and there's a lot about it that's focus on working in a very specific format.

Scrivener was designed by a would-be novelist for prose, not a screenwriter.

I wrote a book back in 2011 in Scrivener, and loved the software for this. I didn’t use about 90% of it but still, it was hugely helpful for structuring a long and complex document of 60k+ words. You need to be able to shuffle chapters around, see an overview of progress and so on - Scrivener does all of this and much more.

I’m dabbling now with writing using Obsidian. I think (but not 100% fully researched / sure yet) that it’ll do all I need: portable, open, easy workflows for shuffling stuff around, exports using pandoc and so on. There are also a fair few plugins and tutorials - Googling “long form writing with Obsidian” will find you lots of useful stuff.

Yes I thin the shuffling around of chapters is the most useful thing about Scrivener.

I'd wish for an Obsidian plugin that stores the chapter order in a file and displays chapter in that order instead of alphabetically.

I'll follow your tip to Googling “long form writing with Obsidian”

>> the shuffling around of chapters is the most useful thing about Scrivener.

Is it better than doing the same in Word using navigation pane to drag and drop entire sections to rearrange? Thanks.

I think so, personally. I wanted the means to see the whole hierarchy (ie book or chapter or both) and be able to move between that view and the editing view easily. Scrivener's tools do this really nicely.

> found a printer in Poland

Would you mind sharing the company name?

It was

https://www.totem.com.pl/

and (some years back) were very happy with the professional service, the price and the result (book had a complicated cover with maps inside and a larger cover that folded in).

I was commissioned to write a book in 2011 - what worked for me given a word count and a deadline was a very simple “I’ll write daily” spreadsheet. I simply divided words remaining by days remaining which gave me a daily target. Main aim was to hit the target but inevitably with writing you have much variety in productivity. So on super productive days I could “overwrite” - which would mean I could feel ok about having a night or weekend off when I didn’t really feel like it.

Stupidly simple but it really helped my nerdy mind. I know it’s not really what you asked but as a technique it helped me take something big and scary and break it down into something paced and manageable.

Software: see comment below, Scrivener or maybe Obsidian?

I'd look into what pandoc can do for you. I don't write books but when I'm writing anything I stuck with pandoc markdown. That way I can pretty much generate any output I like. I occasionally use zettlr too, but mostly stick with ghostwriter.

Seconded! With pandoc, you'll probably be able to establish a workflow where you can go from markdown to epub and also from the same markdown to a web-ready html and a print-ready html. You can then use weasyprint to go from the print-ready html to pdf. You can convert epub into azw for kindle using calibre, or use the epub to get into the various store ecosystems using the proprietary tools that come with those.

Don't get lost in the tools.

I would just go with Google Docs as if you are working with an editor this will make your life a lot easier than trying to teach someone how to send you diff patches of changes and discussing them inline in on GitHub PRs.

My wife writes books. She used mostly Google Docs on her phone and after a few hundred pages the phone app couldn't open her file anymore. We had to open it on a browser, create a new file and copy paste from the previous. I think her edit history was so big the app couldn't handle.

She also tried LibreOffice for a while and one of her documents got to a point where she would open it and it would render just a blank page. I had to unzip the file and extract the text from the xml using a custom parser I wrote. Lost all formatting.

Whatever you do, have backups.

Edit: in both cases everything was just text with very light formatting. No images, no other stuff that could eat all the memory, just mostly plain text.

For my recent book [1] I tried to first stick to Markdown with Pandoc. But Markdown is very limiting for books. Things like callouts, chapters and etc—are hard to do in Markdown.

Therefor I switched to AsciiDoc with AsciiDoctor and I found it to be great. It has support for both ePub and PDF, custom themes, fonts and etc. The syntax is similar to Markdown, so I can use similar tools (Neovim) for writing.

I liked it, and would do it again.

[1] https://www.yieldcode.blog/book/technical-writing-for-softwa...

Honestly I think I'd use latex (but I'm not recommending from self-publishing experience) - as arcane & annoying as it can be... I think trying to do the typesetting/formatting side of it with markdown or similar would be even more annoying? You can do everything you want (processor-dependent, but e.g. mermaid for diagrams is often supported) in markdown but when you want to make it look pretty as a book you're going to need something like CSS, some way of controlling fonts and image/diagram positioning, etc.

I second pandoc. I managed to get a book published with sources in markdown. And I can confirm that you can run from latex, but you cannot escape it.. Before you are done, your markdown will typically end up moving through latex2pdf. I think what nailed me down was proper page numbers. They are one of a thousand small things you expect to be "right" in a book, but nevertheless only latex seems properly prepared to handle..

And as others have said, the marketing/community for a book is a full extra job, separate from being a writer.

It's a short project, but here's an example in Latex+Git(hub)+Pandoc: https://github.com/cosmoosic/wool-june-rud-story

I wrote a longer novella using the same approach, the repo still private as it's in copyediting phase (but it could be opened if there's interest). Using this, I only need to use make to generate both pdf and epub versions (or more formats if needed) -- the makefile has been simply extended with `pandoc -o nl21.epub main.tex`.

Because you can simply split the "main" file and the content, you can just have a short file for different formats -- one for print with left/right margins, one for screens with uniform margins, one smaller for ebook readers, etc.

You can make subfolders to properly organize content, images on one side, chapters in individual subfolders (all depending on how long it's going to be).

In Latex it's trivial to define macros to shorten anything that either you might repeat often, or things like dialog environments. Latex is well integrated with Tikz or other vector formats for diagrams.

Technically Pandoc can go from anything to anything, but I went back to Latex as it's the solution that gives you the best control over your documents while also getting 'out of the way' in all cases where you just want to write down and let the layout be managed by your theme. The syntax has its quirks and the install is heavy, but I find it to be my favourite software stack.

For publishing, I've found Barnes and Noble Press to be incredibly cheap and useable.

If you want the lowest friction, open source, easily extensible Markdown to Web, Kindle, PDF, etc. tool, highly recommend mdBook: https://github.com/rust-lang/mdBook it’s written in Rust, but you don’t have to know any Rust to use it. And then customising is all CSS; for which there are many good (free) themes.

We use it to write docs in our company which are then compiled by GitHub Actions and published as GitHub Pages. The best example of a Book produced with mdBook is the Rust Lang book: https://doc.rust-lang.org/book/

I published a couple of books with the following tools, which I found satisfying:

- Emacs org mode for writing

- Adobe Illustrator for the figures

- Adobe Indesign (text exported from org to IDML via pandoc) for the layout

- Amazon KDP + Ingram spark (aka lightning source) for publishing and distribution.

Be honest about what you are or aren't able to do by yourself. Publishing a book requires a wide gamut of competences: writing, editing, typesetting, graphic design, etc.

One more vote for pandoc.

I’m writing a set of exercise books with photography challenges[1]. I have similar requirements: writing markdown and adding images and diagrams. My output is PDF, EPUB and MOBI.

I started with honkit, I liked that it is a popular and flexible tool. But it didn’t work well for my needs, I was especially unhappy with its default .pdf output.

I switched to pandoc and I am happy with that. I use pandoc-plot [2] for diagrams, it supports plantUML[3]. Also, I write markdown, but the .pdf files are generated through LaTeX and I like the end result much better.

[1]: https://merelysounds.gumroad.com/l/seven-photo-challenges

[2]: https://laurentrdc.github.io/pandoc-plot/MANUAL.html

[3]: https://plantuml.com/

If you're going to do your own typesetting and everything, markdown won't be powerful enough for what you want. LaTeX can do all this for you, or you can use something like Scrivener which is much more of a WYSIWYG tool. I've used markdown for shorter works, and even for them markdown quickly becomes annoying and too simple for my use (especially since many platforms and even the CMS for sites I do paid engagements for don't support vanilla markdown). For longer, printed work with an outside publisher and editors, surprisingly enough Google Docs works really well (I'm using it for 15+ page published PDFs, and authors I know that have written technical books you've probably heard of swear by it), but it will be a pain if you're doing all the formatting and typesetting as well.

I'd recommend AsciiDoctor. It provides a syntax that's very markdown-like. Store your book as src in Git somewhere and move seamlessly between Fedora and your Mac. There's a nice VSCode AsciiDoctor plugin and an extremely helpful community. And it outputs PDF, DocBook, epub, and HTML.

+1.

Also IntelliJ/IDEA is fantastic for asciidoc editing...

You can always create an EPUB. If you enjoy tinkering with HTML, here is a short book on how to create a custom toolchain (disclaimer: I am the author):

https://sigwait.org/~alex/p/pg2e/

I am kind of in the same situation as you - I am no consumated developer whatsoever but happen to be a graphic designer who likes to fiddle with some code and terminal stuff. I've been typesetting stuff with ConTeXt[0], from letters to books to my invoices with ConTeXt + LuaTeX. Some examples at [1]. Nowadays I use vis[2] to do that.

[0] https://en.wikipedia.org/wiki/ConTeXt

[1] https://imgur.com/a/HEzh1x5

[2] https://github.com/martanne/vis

Pandoc permits you to go markdown --Pandoc-> LaTeX --Pandoc-> EPUB/PDF/HTML+CSS.

While you could go directly from markdown to EPUB/PDF/HTML+CSS, leaving LaTeX in the loop lets you benefit from the amazing typesetting knowledge that Don Knuth has embedded in the TeX suite of tools (PDF will look prittier).

Pandoc conversions: https://pandoc.org/demos.html

Pandoc Demo: https://pandoc.org/try/

I use emacs + org-mode + ox-latex (with custom formatting) - then I use bookbaby to print the actual physical books - this is for a textbook I publish

when i was writing https://www.amazon.ca/Developers-Guide-NET-Azure-microservic... "A Developer's Guide to .NET in Azure" , i did it mostly on my Apple Notes app as the chapter structures, moved on to writing in Google Sheets or Microsoft Word. For the code, I wrote it in my IDE and copy/pasted it into my doc. I do agree with most commentary on reviewing using "Tracked Changes" was a little annoying but it was the fastest way I was used to working. I love LaTeX but I can't be bothered to use it if it's not for formulas (university day PTSD lol). I have a co-author as well and a publisher so from a publishing standpoint its the publishers role. I will say there are tons of great self-publishing platforms or if you have a following gumroad, stripe implementation on site and email delivery/auto download.

If you love text editors and terminals, then you should use LaTeX. It is set up to make it easy to show math, but even if you don’t use any math, it’s still going to be your most powerful option. And w your background the learning curve will basically be easy; it isn’t considered that hard to learn by people w your background.

If you need a website version of your book, I would check out Quarto.

It generates ePubs (and other e-formats), a website version, and a PDF.

If you just want a print book, I wouldn't bother with something like this.

https://quarto.org

I'm using sphynx and restructured text. More poweful than markdown. I've never tried laTex, but that is the goto for math. I use mermaid for various charts, it is more limiting than other tools, but the format is more git friendly.

Can you expand on what restructured text offers above markdown that you really like? I've always loved the comparative simplicity of markdown...

You can link to sections in the middle of a document, not just the document. The link is by name so if you move that section to a different document they all update.

Markdown is simpler for sure, but restructured text is only slightly more complex while allowing for more things like that.

For promoting your book once you have published it, it might be worth giving http://boook.link a try. It’s kind like LinkTree but for books.

If you are into Emacs, Org-mode is the obvious option. See: https://orgmode.org/

Google Docs will let you sync Fedora and a Mac and PDF's will provide basic distribution.

That's the simplest thing that might work...or maybe a typewriter + xerox are.

But I digress. Good luck.

Pandoc works pretty well these days. I writes in LaTeX and my Professor works in Word. The conversion works just fine.

Are you looking for a tool professionals write books with, or a CS tool that can be stretched to write a book?

The professionals use Microsoft Word.

Well, some of them. Others use Obsidian, like P.D. Workman, who has written 43 novels in Markdown, many of them with Obsidian. [0] And a few still use WordStar and WordPerfect.

Word is indeed the lingua franca of publishing. To go to an agent or publisher, a doc or docx file is where a manuscript usually has to end up. But "end up" is important here...95% of the work of actually writing need not happen in Word, and many successful writers don't use it for primary drafting.

[0] https://pdworkman.com/writing-a-novel-in-markdown/

R Bookdown is really good for going from markdown to book.

https://leanpub.com

EDIT: why downvoted?

Sigil?

Reply on news.ycombinator.com