Writing a book, still the same pain 15 years later | Hacker News Reader

Writing a book, still the same pain 15 years later

medium.com

150 pointsmpweiher81 comments

I’m a working novelist (I put out about two books a year, I’m a WSJ best-seller, Edgar nominee, blah blah blah) and the number one mistake I made in the past is the same as when I would sit down in front of a code editor or Photoshop and not have a plan. While we can all “seat of the pants” things to some degree, without a detailed plan your chances of getting derailed increase exponentially.

I’ve written best-sellers in week (for real) - not because I have any special skill (I’m not a particularly fast typist) but because I write very detailed outlines that break things down enough that I never stare at a blank screen yet always have enough room to have fun improvising.

Outlining for me is asking a series of nested questions about “What” and “Why” and always knowing where I need to go next.

Just curious, how long does the outlining take? I've read a similar notion before (Doctorow claiming to write a book in a year by writing 20 minutes per day - but he also has a plan what to write beforehand).

I am curious if outlining doesn't really count as work in the minds of many writers?

I spend anywhere from a few days to a few weeks working on an outline. This is while doing other things.

How do you writer outlines. Is there a process. Or are you just jumping from idea to idea? Is there any good guide on how to come up with them?

First I write a couple sentence mission statement about what the book is about, the character’s journey, the resolution, etc. This is my way of keeping track if the book is on course or not. Does every chapter serve that mission?

I then start with notecards and begin with the big questions: What’s the major conflict of the book? What are my main character’s conflicts? What are the conflicts of each act? I then break these down into chapters. Every chapter has a smaller conflict and resolution.

On the chapter break downs I ask what the character is trying to accomplish. Does it work out? How does the outcome change the story?

When I do my chapter-by-chapter breakdown the characters are driving the story - this is what makes it fun for me to write. Instead of me pushing a character from point A to B, it feels like the characters are driving the plot.

Using this approach I almost always redo the last 30% of the outline when I get to it because the characters have made smarter choices than I the writer had planned. I’m very into writing from a Theory of Mind approach.

Let me reiterate that while this approach works great for me, it may not work for everyone. Some people love to write without outlines because they just want to jam on the keyboard. That’s just as valid as any other approach - although I’d like to point out that just because a pro writer says they don’t outline doesn’t mean they don’t have an innate sense of structure in their head. We often don’t know what we know.

Thanks a lot, appreciate the insights. I always find the productivity part part the most challenging of it all. That's why I am sold on the structure approach and not waiting for the kiss of the muse.

I've done a few Periscope streams on my Twitter feed about this. You can check them out at @AndrewMayne.

It's worth mentioning, perhaps, that a lot of writers don't have a plan/outline: Stephen King, for example, who has described in great detail how he works in "On writing": he starts with a situation and develops it as he writes. In fact, in most cases where I've read about how a writer works there is no plan/outline. However, it was mentioned in court proceedings that Dan Brown had an outline for "The Da Vinci Code", so I don't have sufficient evidence to conclude which approach is most popular. Perhaps it's just those writers who like to write about writing that prefer not to have a plan.

“On Writing” is a wonderful book. If you’re Stephen King, plot and structure run through your veins. If you’re a mortal like the rest of us and “seat of the pants” isn’t working...then outline. Outlining also saves you from having to randomly kill off half your characters in the middle of the book because there’s no way to wind down all their storylines...

I’ll also add that anecdotally, my writer friends who outline go through far fewer revisions than those that don’t.

If your process works, don’t mess with it.

Writing a book is easy. You sit down by the keyboard, slit your wrists, and pour your self out. (That's not a typo, btw.)

I've written one, and in the process acquired a near limitless admiration for those who can do it for living, book after book, decade after another. Even for a topic you feel should be easy - every page that ends up being in the final result, you have written at least 3x. The editing process alone is brutal, and the harsher, more demanding editor you get to work with, the better the results.

I can say the same. I self-published a 500+ page book last year and was so drained from the process I haven't even seriously thought about writing my second book.

When your armchair fan is sitting back criticizing George Martin for not finishing his books, I empathize with him. I totally get it.

I allow myself to criticize Patrick Rothfuss though.

The absolute arrogance he displayed after publishing the first book of the series was astounding. He claimed the series was "already finished" and that each of the other 2 books would be released a year apart, simple as that. He wasn't like other authors, he assured us.

Here we are, 10 years later with Rothfuss literally doing everything except finishing the series (making tv shows, starting charities, making video games, board games, etc.)

My theory is that he painted himself into a corner and doesn't know how to finish it without it being a train wreck.

People also just lose interest in things. And they'll find just about any excuse--"The doorknobs really needed polishing!" "There's a mistake on the Internet!"--to justify pushing off working on some project today. Days turn into months and years.

I hate all the criticism Rothfuss gets. He's had a version done for ages, but in his words it was “a book you would not have liked, because it was just discernibly bad.” He's spoken at length about the challenges and I find it hard to hold it against him. I'm sad Doors of Stone isn't out, sure, but he doesn't deserve any criticism.

He painted himself in a corner very clearly. The series is about the legendary chronicles of a kingkiller, but 2/3rds of the way through, the story was about a street urchin, then a magical academy.

I thoroughly enjoyed the stories, but I feel closer to Harry Potter than the kingkiller. If he tries to get there in one book, I believe he will stumble. The tone and scale of the story so far has been much too small for a trilogy that promised an epic finish.

I'm so bummed about this. I discovered (and inhaled) the first book over the weekend, couldn't put it down. Looked the author up, and I was so conflicted because I liked the book as much as I dislike what I read about the author. I hope he figures out a way to sort things out. I'm actually very conflicted about starting the second book after reading about him.

As a side note, don't think writing a second edition makes the process any easier. I recently finished the second edition of my book on technical blogging and I'm left with the distinct feeling of having "changed little" while objectively having spent more time writing this than the first edition.

Still, in the end, you get to hold your baby.

As Winston Churchill said, "Writing a book is an adventure. To begin with it is a toy and an amusement. Then it becomes a mistress, then it becomes a master, then it becomes a tyrant. The last phase is that just as you are about to be reconciled to your servitude, you kill the monster and fling him to the public."

Back when I regularly wrote white papers/explainers/etc. of various sorts (3K-4K words or so), I often felt a need to redo a piece that was 12-18 months old to fix a couple things that weren't correct any longer, shake any dust off, and so forth.

Invariably, half the thing ended up being rewritten to various degrees. It was still a lot easier than starting from a blank sheet of paper but, as soon as you rework something, you don't want to ignore all the sections that aren't quite fresh, want to bring in new facts and figures, bring in more recent examples, etc.

And, in reality, the extra time is probably justified because the act of opening up an older piece, republishing it, and all the associated background machinery makes it worth spending a bit more time.

> As a side note, don't think writing a second edition makes the process any easier.

Why?

Just add "dot" on last page -- and second edition of your book is ready!

No, no, no. You have to move the chapters around, renumber the sections, change the order of the questions at the back, and introduce a couple of new typos.

Otherwise how are you going to force the students to buy the new edition?

FWIW, the numbers in that piece seem pretty spot on for me.

Something around 75K-100K words (translating to about 200 pages with some figures, etc.) is probably the minimum for what publishers (and most readers) consider a "real" book.

I've been writing most days for about 20 years. I've always considered 1K words/day a good ballpark for a "writing day." (That's a decent length for a blog post or article.) That's with various meetings and other activities/tasks but without any other major commitment. (i.e. I find it hard to crank out 1K words at the end of the day after getting home from a day of work or an event) I'm considered a fast writer by people I work with.

Do the math. And that's more or less ignoring rewrites, extensive edits, any in-depth research/interviews, and so forth.

I've tried publishing a novel in India (English) and it is much more painful than the 80k limit.

A top editor told me, more like lamented, that they focus on historical fiction or romance because "that's what sells"

I wrote a decent thriller which my friends read in one day because they loved it. Granted, they are my friends so they could by lying, but the publishers just read the synopsis and saw that it isn't historical or romance and ignored. That's what Indian publishers do, they ignore!

And there was this one agent who asked me money so he could "represent me". He then wanted me to sign a contract, which meant that if I didn't publish book to publisher he would get then I'd have to pay him 10k or something. And his mother has a publishing house. I read online that he gets books published from his mom's house.

Which agent asks for money?!

I have 4 unpublished novels, one is about a guy who wakes up at a bus stop with 0 memories and tries to figure out his past, one long fiction about science fiction that spans 1000s of years and one is a satire on the new form of Authoritarian regimes. This satire is a 2 book series.

Have you considered self-publishing? If the books are complete and would be wasted otherwise... may be worth a try.

I have. But I'm still researching how to market the books. I have 0 experience of marketing of a book

First rule of publishing: money flows towards the author, not the other way round. (Not necessarily very much money mind, but it’s the direction that’s the important bit!)

No reputable agent charges authors directly for their services - their income comes out of a cut of the money paid by publishers to authors.

If you have a stack of books that your friends actually like & can’t get published, do you have anything to lose publishing them yourself on Amazon’s Kindle platform?

First rule, that's what I thought and I declined his offer.

You won't believe his arrogance. He replies "This wasn't an offer"

I was planning to publish on Kindle but I'm not sure how to market the book.

Do you know perchance how to market Kindle books? I read that 99% authors don't market their books.

I want to be among the 1%!

Definitely not my area of expertise! But go read the blogs of people that have been successful at it - plenty of tips there.

Thank you!

You could write a book which is about 4 fictional books that never got published :-)

Not sure who would buy such a book

It might be interesting. And maybe a common theme would emerge why those four fictional books got never published. It might be a bit like "Cloud Atlas" where the different stories are connected in a way that becomes apparent only in the end.

Consider that none of the sub-stories of Cloud Atlas were good enough on their own to be published (and turned into a movie).

I deeply regret writing my novel. It took an agonising 8 years when I was desperate to get it out as soon as possible. The issue was that I wanted a certain level of quality. I ended up self publishing on Amazon :/

I try not to think about what I could have done with the time: study math, programming, do stand up comedy etc. etc.

It is what it is I guess.

I understand your pain, but I want to ask: did you put much effort into marketing it after publishing? A lot of writers put 99% of their effort into writing, and by the time they've finished the book they're out of energy to pursue the marketing aspect. This is compounded by the fact that talented writers are not necessarily good at the marketing side of things.

The key is to persist with marketing until it either succeeds, or you've got clear evidence that most readers don't like it enough. If you're not good at marketing, find people who are.

Sometimes, the key to success, especially if you've got a quality product, is to persist after most people would have thrown in the towel.

Would you mind sharing a link to it?

At risk of sounding like a low-effort critic, I have to say I'm not sure what we're supposed to take away from this post. It's got about zero useful information or interesting insight.

It really does come across as "How not to write a book" but without any sort of...life...to the writing.

I say that with nothing but respect and admiration for what Kent has contributed to the world of software engineering. As for this blog post in isolation, I just can't figure out who would read this blog post and feel it deserved to be shared around.

I think it's written for other writers. To help communicate the pain and process. It's easy to feel like it's "just you" and everyone else just shits perfect chapters, one per week, and hits publish.

I enjoyed it.

There's an industry of writers providing advice to other writers about writing. Think of it like self-help but for writing.

I used to attend some writers groups, but everyone was always looking for a method that would make writing easy, instead of just writing.

Almost nobody knew what they were talking about. Figuring out what works for you is writing.

Presumably, the people who actually know how to "operationalize" writing, aren't writing self-help books, but rather are managing stables of prolific (if maybe schlocky!) writers.

I would love to hear writing advice from the managing editor of a publisher of pulp serial-fiction books, of the kind where the writers churn out multiple new books per year. Those books might not be the greatest thing ever written, but certainly they're better than failing to ever publish anything.

For that matter, there are series of branded tech books of various sorts that are used by companies for marketing campaigns. They're written by freelancers and are often more successful at their objectives than you would like to know.

I've always been curious how that scene works. It feels like a comfortable place for someone with broad curiosity and an ability to write. Do you have any insights?

That's the joke right? Even if you could tell me exactly what worked for you, it wouldn't work for me.

If there was a formula for success, we would all undertake it.

There is No Silver Bullet.

Hey, maybe someone should write that book.

As in software, as in writing, as in life.

Let's, co-author :)

I didn't find it so much "How not to write a book" as a description of what writing a book felt like. Maybe there are people who have systematized the writing of a certain type of book to the point where it's just a routine few hours they put in every day. But I think the experiences expressed in this post are more common.

It seems to me it's to let prospective authors know what they'd be getting into. To provide them with realistic expectations.

The post says "To help you quell the desire to write a book of your own, here are the steps."

I think there's an interesting phenomenon here how a lot of people get obsessed by producing a book. I have seen so many blogs by aspiring writers, and overheard their conversations on trains, too. What drives people to do this? I think most of them don't hold much hope of becoming rich or famous: they understand that most writers need a day job. Perhaps for some reason they consider themselves inadequate until they are a published author. But why? Does it just follow from the way authors and publishers create a mystique in order to sell their wares?

The mystique (if that's what it is) seems to be quite specific to being the author of a book. Writing a series of articles for a national newspaper or being a commercially successful screenwriter or ghost writer somehow isn't good enough, even if it's much more lucrative.

A book is something that you can point to and say "I made that". Regardless of the content, a book is not fleeting or ephemeral. These days people will even look up to you if you are capable of reading one. So it carries weight, both in the physical and mental sense of the word. If you're not self published, a book is also a sign of approval. Someone who knows about these things thought it was good enough to print. Really, there are lots of reasons to want to write a book.

That is true. Books are read after centuries, magazine articles not

This is a really accurate account of what it's like to go solo on a book project. It can be brutal.

I'm surprised and a little sad there isn't a full discussion of how to line up friends and admired colleagues to make the journey easier. They can help with the early brainstorming. The best ones are invaluable advisers on scope, pacing and cadence. They will remind you about the merits of your best ideas when you feel lost, and they will help you cull with a lot fewer tears.

I tried doing my first book solo and stalled out after a year. Rounding up an advisory circle let not only brought that book back to life -- it also created a trusty new process that has let me write four more books more quickly and with less stress.

For those who write eBooks, what is the best output format? I am planning to write a introductory book on deep learning which is going to contain a lot of math and code. I find LaTeX as the best way to author math stuff but I don't know if I can generate .mobi and .epub files from it.

Are the .mobi and .epub formats even worth it?

Would it be simpler to just output an A6 format PDF that can be viewed alike on desktop as well as eReaders?

I use pandoc+xelatex to convert GitHub style markdown to pdf [1] I've also noted down other options in the last section of the blog post, which I think should get you started.

I haven't yet been able to generate a decent looking epub/mobi for technical books with code snippets. And these also depend on which reader user has. I've been asked by a few to support these formats, but majority are happy with pdf.

[1] https://learnbyexample.github.io/tutorial/ebook-generation/c...

Thank you for sharing your insights. Can you share more details about what issues you with epub/mobi for generating technical books? Do you have any experience with machine learning and math content? If generating epub/mobi for technical books is a problem I am guessing it would be even more of a problem to generate good looking epub/mobi with math content.

Issues with epub/mobi: pandoc gives me error which I didn't put much effort to get it resolved. I tried a sample book generation using rstudio and the resulting epub was bland compared to pdf (no syntax highlighting for code, spacing isn't pleasing between code and text, etc) - I searched a bit and found that I'd need to know learn a lot about CSS to get better results. A book from another author on leanpub (the platform allows you to generate all types from single source) was remarkably poor to read on my desktop epub reader compared to pdf (again no syntax highlighting, spacing issues, etc). Do try to compare same book in pdf vs epub/mobi if you get the chance.

No experience with ML. Have read a bit of content for Python+Math, but only as pdf.

I'm doing a book project now and thought I should try to escape LaTeX and went with HTML/CSS paged media through weasyprint. Don't do it. The technology is not quite there yet. Go with LaTeX.

Maybe you should look into nootropics, nutrition, and meditation.

In my case, I'd have to let everything sit on my mind (refine, enrich, blend, blur, cohese..) for a while. After, I'd write it all in one go.

Outlines are sometimes great, as they allow pacing and regularity. On the other hand, they tend to destroy substance in a work.

Unless writing for leisure, ensuring there's a sufficiently-sized accessible interested market for your topic.

The more effort put into a book/novel, the less likely it is to do well. Many books catch on due to a good title (picking the right/hot topic). Everything else mainly involves revolving around and repeating the desire leading to the hot nature of the topic. The editor will handle the rest (grammar, spelling, etc). Most who buy books are women or feeler males (ie, all emotion).

"Smalltalk Best Practice Patterns" was a good book

s/was/is/g

One of the biggest insights I gained was reading about "intention revealing selector".

Before that I had thought that we needed "higher level" programming languages, whatever that meant, exactly.

This made me realise the importance of naming (one of the two, three? hard things I computer science). If I could find an intention-revealing selector, I didn't need any higher level, because the current level was just fine.

This was the case much more often that I would have thought, it just required work. (eek)

And it also gave you an indicator that you might, in fact, need a "higher" language level, or more accurately an architecturally more appropriate one: when you can't create intention-revealing selectors.

If anyone else was wondering: http://wiki.c2.com/?IntentionRevealingSelector

> Name methods after what the method does, don't name methods after how it works.

This rule applies even more to API documentation. As soon as you mention how, that becomes part of the what interface you have to support.

With internal method names, for tricky code, it's a bit more fuzzy, and sometimes I've found that putting how information into the name seems to be a great idea, for readability&correctness of code that calls it. This is more for functions/methods for which the how is relevant (or, you could say, some of the how becomes part of the what).

What vs. how was what made this click for me.

Apparently, there was a method to invert the screen

   >>>flashScreen
      self invertScreen.
      self delay:sometime.
      self invertScreen.

There was also the requirement to have a "visual beep" to accompany an acoustic beep. The way the visual beep was to be implemented was to invert the screen.

Now you could have just put "... invertScreen." where you were supposed to do the visual beep.

Or you could add a method:

   >>>visualBeep
       self flashScreen.

Before, I would have said this method is completely superfluous, after all it doesn't really do anything that isn't already in the system. However, the service it provides is giving something the correct name. This is highly valuable.

Yes, it packages a set of code-lines together which have the shared property that they all serve the same purpose. Even if it is called from only one location it makes sense EVEN THOUGH IT DOES NOT REDUCE the code-size. It does the opposite if you call it only once. But it makes it easier to understand the code because it hides details under a separate method. It makes it easier to read the calling code because it makes the calling code shorter.

I'm not quite sure however if this was the reasoning behind Intention Revealing Selector which I thought was more about how to name methods, not about whether and when to package multiple method-calls into a single new method.

I think the problem is totally different for those who live to see their projects finished. Then there are people like me who have no such urge.

If a person like me can get into the routine of writing (which would be very difficult trick) then the writing itself would probably be quite easy. There is no fear of blank page, no depressive thoughts of "am I ever going to finish" etc. But there is no reward in the other end of the tunnel either. The writing itself would have to be fun enough.

What you describe seems more akin to why people write diaries rather than write books.

I'm sure I'm over-generalizing but anyone who sets out to write a book has publishing it in mind--and, if they're using a publisher, the publisher certainly does.

Things like blogs blur the boundaries to be sure. But I still view blogs as discrete published items rather than a process I enjoy for sake of the process.

For me, photography is probably more in the enjoying the process camp although I do publish it as well.

I am working on my first book and today just cut it into two books because ... because it demanded it.

But doing 100k words? wow? And getting early feedback and marketing and ... lots to do :-)

This means writing a book is a creative endeavor.

Creativity does not get easier with time.

These discussions about writing books are the best thing in Hacker News.

(Aw, I was hoping this would be about publishing a book with LaTeX...)

I myself write books in Markdown. Each chapter has a markdown file, the book is version controlled with Git, there is a "build script" that does a "build" (in this case it's a shell script that concatenates the markdown files and runs them through pandoc and calibre CLIs to generate eBooks and PDFs).

When I send to my editor I just tell her to modify the files as she sees fit. When I get them back I put the new files on a branch and create a PR so I can start discussions on changes, etc.

I think the process is pretty darn efficient but it's hard to sell it to non-techie writers/editors.

I wanted to use a workflow like this years ago working with Apress. Possibly (hopefully) things are better now, but at least at the time, all their tooling expected Word docs as the input, and you had to use a very specific stylesheet (including goofy stuff like magic text strings to insert images).

There were also some manual transformation steps occurring on their end, because there was a cutover point where the Word doc was no longer accepted and you had to just quote page and line numbers to get further changes made.

Word docs, Sharepoint. (Though I didn't have that experience with images.)

> There were also some manual transformation steps occurring on their end, because there was a cutover point where the Word doc was no longer accepted

This was probably the point at which the text had been moved into InDesign for typesetting by the designer.

Are there publishers who are more accommodating of inputs like LaTeX/pandoc/markdown/etc? It'd be great to know that kind of thing when considering publishers.

You should check out LeanPub. They support a variety of formats, and do some really nice things if you want to self-publish.

I think No Starch Press accepts LaTeX markup. There is a nostarch package on CTAN. https://ctan.org/pkg/nostarch?lang=en

The content of their books is often OK, however low-res images,pixelated tables and some random font and its size makes those books look funny..

Would you mind sharing this script? I am moving all my notes to markdown and I would like to setup something like that

Seconding this, for the same reasons really.

It's more of a haphazard set of custom scripts I'm afraid, they are pretty messy. I really should clean them up and host on GitHub or something. But, just to get you started, here's one of my scripts (a batch script I use to generate a kindle book on Windows):

    :: Convert Markdown to DOCX format, adding a pagebreak at every \newpage
    pandoc -f markdown -t docx --filter=pandoc-docx-pagebreakpy -o build.docx ^
        foreword.md ^
        preface.md ^
        acknowledgments.md ^
        part_a.md ^
        1.md ^
        2.md ^
        3.md
    
    
    :: File needs to be opened in Word, saved, and then closed to generate physical Table of Contents :/
    @echo off
    echo Please open build.docx in Word, save, and then close.
    pause
    
    :: Set e-book metadata
    ebook-meta build.docx --authors "First M. Last" --author-sort "Last, First M." --title "Book Title" --title-sort "Book Title"
    
    :: Convert DOCX to MOBI (Kindle)
    :: --share-not-sync Makes it so cover shows up in Kindle for PC but also breaks Kindle WhisperSync. Remove if you want WhisperSync.
    :: See https://manual.calibre-ebook.com/faq.html#the-covers-for-my-mobi-files-have-stopped-showing-up-in-kindle-for-pc-kindle-for-android-ipad-etc
    :: --cover "media/cover.jpg" --share-not-sync
    ebook-convert build.docx build.mobi

I am using the same setup though with a lot of customization of pandoc

https://github.com/boazbk/tcs

Oh cool! I'm getting reasonable results using Pandoc->LaTeX->PDF with mostly markdown and bits of raw LaTeX as the source.

I pass `-V documentclass:scrbook` to pandoc and then I can use e.g. '\frontmatter' and '\mainmatter' in the MD source.

As I say, I'm getting reasonable results, but it's one of those "I can't be the first person to do this" situations. Got any tips or links to share?

I read this three times and still don't understand. What was the point or thesis of the article?

Reply on news.ycombinator.com