The most important quote here for me:
"What kept me from [writing my own tool] was a calm voice in my head telling me that I’m here to write a book, not a preprocessor. [...] Now I have written two books and zero tools, which I consider a success."
"What kept me from [writing my own tool] was a calm voice in my head telling me that I’m here to write a book, not a preprocessor. [...] Now I have written two books and zero tools, which I consider a success."
Many people have problems focusing on the task they set out to do. It's a type of procrastination. Insert that famous gif from Malcolm in the Middle.
This is relevant to startups as well (and business in general): Being able to decide whether something is in scope of a project or not. You are encountering a roadblock, how do you react? Do you set out to solve it perfectly or do you work around it (or even, do you give up)?
The quote you pasted is the perfect way to put it into words. Another common theme I hear about is people asking: "How do I become a successful [profession here]?" (eg. "streamer", "youtuber", "musician", "programmer", "writer", ...). The answer is always the same: "Start".
I don't agree at all. It seems to me that the most interesting advances of mankind have happened precisely when people were trying to do something else, and had to invent new tools. The tools turn out to be more important than the final result because they can be used by other people.
If you're trying to write a book, and you want to write a book, write the damn book. There's an infinitesimal chance that the tool you're writing instead of your book will change the world.
If you're trying to write that tool, and want to write that tool, then power to you, but don't lie to yourself and tell yourself it's "for the book". Just admit you're switching focus.
A close friend of mine has been trying to create a game for .. a decade or so. Every time he starts, there's always something. Instead of using, say, Unity, he tried his hand at writing a game engine. A physics engine. A programming language. Because none of the tools are "quite right". Of course, none of those get completed either because they're a shitton of work. And his work didn't change the world, because it's never completed. At the end of the day, much like many in the software engineering world, he's a brilliant mind busy optimizing ads. And he doesn't have a working game engine, a working programming language, or a game.
I don't want to sound demeaning, because it's still very good experience to acquire. And in general I don't want to tell people "don't write your own a programming language". I think these things are super cool. But then if that's what you set out to do, make that your project instead, it'll have a much higher chance to actually complete, instead of sitting in an incomplete state, in a hidden unfinished repo with nobody to share it with.
The common thread between all of mankind's most interesting advances: They're finished. How do you finish a project? Well... start.
> If you're trying to write a book, and you want to write a book, write the damn book. There's an infinitesimal chance that the tool you're writing instead of your book will change the world.
Not the person you were replying to, but their point certainly rings true to me, from all the history I've read or otherwise heard of over the years. They could just as easily ask you for citations.
"if what you say is true, it doesn't change anything at an individual level" -- yes it does. It's usually individuals that come up with the important tools and inventions. Those development are very significant because they make systemic advances.
"There's an infinitesimal chance that the tool you're writing instead of your book will change the world" - it doesn't matter how small the odds are for any one individual.
There's two points here. One is that the effects can be cumulative -- even small advances and bits of knowledge gained contribute, over a longer period of time, to large benefits.
The other is that the historical reality is that tools that were developed as side effects of other goals have made a very significant overall effect, and this can't be denied by a blinkered focus on the probabilities concerning individual cases.
Tools are super valuable; it's arguable that many of the things I list above are sort of tools. I just don't think that a tool that is quickly slapped together in the service of another project is likely to be a game-changer.
In fact, I'd say that one of the keys to growing your personal impact is to find the best work that's already been done, and use it to further your goal. Even Newton said he was standing on the shoulders of giants.
Then there are the three branches of modern mathematics, according to V.I. Arnold:
1) modern hydrodynamics and PDE, are just guys procrastinating while they are paid by the manufacturers of nuclear submarines
2) celestial mechanics, control theory and ODE, are just guys procrastinating while they are paid by the manufacturers of rockets and missiles
3) number theory, are just guys procrastinating while they are paid by the CIA to develop cryptographic tools
agriculture : this kid is just playing with small seeds instead of collecting nuts for winter
computers : just tools to speed-up the important computations of nuclear reactors
TeX : the guy just had the write a damn book about algorithms but spent 10 years procrastinating
Bach : the guy just wanted to glorify God
I stand by my point that nearly all important advances of mankind are tools developed to solve a long forgotten goal.
That is indeed true.
Of course, almost all of the goals I have accomplished for myself have been accomplished by actively engaging in pursuing that goal rather than pursuing some side channel, while the side projects I've created have generally been a lot less useful to myself and humanity than, say Bach. :D
If we just open up the idea of "tool" wide enough, everything becomes a tool. Writing a book isn't actually a goal, it's just a tool for acquiring money and fame. Calculating the orbit of planets was just a tool for Newton to establish social credit and historical significance. Agriculture is just a tool for eating, but eating is just a tool for living, but living is just a tool for procreating, but procreating is just a tool for perpetuating genes.
Why are we perpetuating genes? Really, no one knows. Solve that problem, I guess, and finally we'll have the one project enriquto can't easily dismiss as tool development.
I managed to write a 25,000 word story in a word processor once, but it was such a jumble that I just closed the file and forgot about writing for a few years. I know now there are ways to organize with folders and spreadsheets, but now I do most of my writing on the phone. Scrivener's binder makes that easy on a small screen.
(Author here.) My point was not that you shouldn't invent tools if you need them. By all means, go ahead. My heart beats for creating tools (especially tools to create other tools...).
What I wanted to express was this: if you want to write _a book_, first write the book and only then worry about the tools. If you then notice that you need a custom toolchain to get the book in exactly the shape you need it to be, do it.
I've had people tell me that they would love to write a book, but, well, the existing tools are not sufficient enough. And I think that's the wrong approach to writing a book: write something and you'll quickly realize that the tooling is not your biggest hurdle.
That’s hilarious. Humans were hand writing books on paper and giving them to other humans to hand write copies. A good craftsman never blames his tools.
Not very many were, and a big reason for that was the available tooling.
> A good craftsman never blames his tools
Someone who writes a book by blames the available tooling support for the low quality of its content is a craftsman (but, per the saying, not a good one) blaming his tools.
A person who chooses not to write a book at all because the anticipated opportunity cost given the existing tooling support and that person’s personal utility function is not.
I agree 100% with this. Following the case of Knuth/TAoCP/TeX, in fact the first edition of TAoCP was not written in TeX, he made TeX to improve his already written book.
For me, that time started about two years ago. At 31, a full 25 years since first learning how to program[0] I finally felt like I could be trusted to make the right calls on when to invest the time to abstract or create generation or code parsing tools; and when to do things the old fashion way.
After two years of spending about 20 to 40% of my time on tooling I'd wager I'm about 10x faster than I was at 30 and about 20x faster than I was at 25.
The problem now isn't writing the code. The problem now is that the code is so fast that I don't have the same amount of passive time to think about features / strategy. I have to intentionally set aside time for it and I find it unnatural. Also, I know too much about cybersecurity which is good for my end users, but bad for my feature productivity.
[0] Poorly. My mum taught me how to code QBASIC. I altered video games.
Indeed yak shaving should only be done when one has the breathing room to do so. It's just good common sense, except there are exceptions. Slack was a yak shaving effort. The most valuable things I've shipped have all been yak shaving efforts. I guess I'm just better at building tools than anything else. Perhaps I enjoy it more. shrug
To each their own, but if you're trying to ship a product instead of a bald yak you're going to have to do it the ugly way, with tools you could fix, etc...
That recursively applies to each tool used to make the product as well.
I think I’m going to frame that and put it on the wall. Did you come up with that or did you hear it somewhere?
Before:
I'll get my clients to give me access to an S3 bucket that I can push their data to.
Now:
I'll make sure I get my clients to give me a password or public key to encrypt their data before I push it to the S3 bucket because many people fuck up security on those things.
To my end user, it's obviously more secure, but it isn't lining my pocketbook and most end users don't care about security, even if they should. Same thing with security headers or anything, really. Security slows you down at some point.
For one thing, at the time, Unix and all its associated tools were really not prevalent outside of Bell Labs and a few places — troff would not have been available to run on the computer systems Knuth had available to him in 1977. (The ones at SAIL, which were PDP-10 machines running either TOPS-10 or WAITS, I forget what I read.) He didn't have the source code of troff either; all he had was published descriptions: and those he did use, e.g. the paper by Kernighan and Cherry describing `eqn` came out in 1975 and he used ideas from it while designing TeX's syntax for math.
For another thing, troff was designed to control line printers and other such devices that were typically attached to computers; it simply wasn't designed to drive a digital typesetter of the kind Knuth was thinking of (Alphatype CRS). So he'd need a new program anyway. In fact, he says that until he saw Patrick Winston's book on AI in 1977 (which was one of the early books produced on a digital typesetter, i.e. where the actual technology for getting ink onto the page was based on 0s and 1s, instead of being based on either pieces of lead (hot metal typesetting) or light and lenses (phototypesetting)), he had simply not associated output from computers with “real books”. Things like troff were designed to control the simplistic devices that came with computers, that could move by discrete (large!) amounts on the paper (e.g. some could move by full “lines” only), and strike down specific symbols (from a limited font repertoire that came with the device) onto the page. This wouldn't have seemed relevant to Knuth, who primarily wanted to create his own digital fonts (as he did with METAFONT), and TeX was among other things simply a way to use those fonts.
So, regardless of whether one considers troff to have very beautiful typesetting or not, it simply wouldn't have seemed applicable to the problem at the time, to be rejected as "not good enough" or not. Now of course with output going to generic page-description formats like PDF (even TeX's DVI was not Knuth's idea—he was thinking TeX would just write code for the device directly—and he had to be convinced to do it by Fuchs: https://tug.org/interviews/fuchs.html), and high-resolution printers (fairly high DPI, even if not 5000+ as with the CRS then) available to typical consumers, we don't appreciate all this. :-)
https://www.amazon.com/Computers-Typesetting-B-TeX-Program/d...
Of course, that's the ideal. I can't use my preferred setup when I'm working on our legacy Windows codebase. I also won't pretend that I'm immune to chasing the new shiny thing to distract me from doing real work, it happens with some frequency. I just think that editors (and perhaps a good tiling window manager) are the exception(s) to the rule.