Build tools around workflows, not workflows around tools
thesephist.com
thesephist.com
And if you ask how they're different so that they'd need a unique process to manage bugs, to manage invoices, to manage inventory, you get pretty much the same answers across the board. You hear about personalities and power structures and anecdotal preferences.
And sure enough, that's exactly what you see at the link: Each person’s mind works a little differently, and each person remembers and processes information a little differently.
Yes, you're unique. Just like everybody else.
The fact is, the great tools that have achieved widespread adoption understand the problems, and all their caveats, better than you. Picking any established tool to manage information and workflow and getting good at using it will usually yield better results.
The search for the perfect tool or the perfect workflow or the perfect process always ends up being the enemy of good.
Another way to say it is instead of taking someone else's blueprint, we build our own blueprint and use pre-fabricated parts to realize the blueprint.
In the end, sometimes solutions look very similar but there is that one little bit that gives you a competitive advantage that no one else has.
I'm all for doing something "different" if it gives you some kind of core competitive advantage.
But the pain of being off the beaten path can really exceed small benefits you get in unexpected ways.
Save real innovation and risk for the things that are core, unique to you, and possible sources of large competitive leverage. Be boring elsewhere, maybe with an occasional small sprinkle of something clever mixed in. Evangelize the small bits of cleverness, in hopes that someone else will start maintaining it. ;)
Ideally, I map out workflow and find tools that fit the workflow in the Unix philosophy. Well-defined, do one thing well. As opposed to monoliths that purport to do everything.
My experience has been that doing things in this way builds a system that is extremely robust to significant changes.
That's not to say you should always roll your own. This is one of those things that's really hard to become good at: Decide early on whether to go with an existing solution or create something from scratch. (and if you go for an existing solution, which one?)
There's many arguments for both sides, one thing often mentioned is that settling for some popular solution or framework will make it easier for others to get into your project, but if you bend over backwards to get three different tools to do what you want instead of writing a hundred lines of bash, you might be doing something wrong.
I thought the same way except I was even more insane. I thought lets just teach everyone English and we won't have to do any internationalization/localization.
But seriously, in a lot of cases, translating and localizing is actually the easy part of internationalizing a business.
In my defence, I was not a very travelled child. I didn't even know about all the different kinds of power socket standards around the world.
The stagnated companies with questionable data analysis because they're looking at beautiful SAP or other ERP graphs that have been filled with incomplete data by their end users on awful interfaces would disagree...if they knew.
Most of any business is commodity stuff, typically things like HR, payroll, logistics, finance, security. Except if your business is one of those functions. Take Amazon's logistics for example. They aren't just any online retailer, they are the dominant online retailer in large part because they optimized the hell out of their logistics pipeline.
But if it's not your bread-and-butter, then yeah, go ahead and change your workflow to whatever COTS software you pick requires. You'll be fine. I worked for an insurance company, and they picked PeopleWare for their HR system and used commodity software tools for their actuarials. I worked for a very large shoe company they has SAP at the core of their (extremely complex) product lifecycle management system because their real business differentiator is their design and marketing.
That same insurance company had a customer sales pipeline tailored to a specific market – high-end professionals like doctors, lawyers, and similar – so they had custom software for that. That same shoe company had customized design and bill of materials systems so they could get their shoes to market quickly and globally.
Rent for parity, buy or build for competitive advantage.
Of course now that everything is under pay as you go SaaS model, that's out of window.
Still in Europe it is legal (http://curia.europa.eu/juris/document/document.jsf?docid=124...) and people do resell mundane licenses such as Windows.
Driving instructors typically have their cars on a 3 year hire purchase then return instead of buying. Why? Way back when, my driving instructor actually bought one of the cars. 3 months later, the engine failed and it was out of warranty.
Because of that, he realised that paying a monthly fee meant he had no surprise failures, he returned the car just as the clutch was burning out (learner drivers don’t have great driving technique, weirdly enough), and had a fresh car every 3 years for new learners.
Only build if you have the talent to do so. Early on Amazon managed to attract serious tech talent to the company. That enabled them to build. If a company can't do that then it's better to buy no matter what.
I would certainly hope that if the function is part of what differentiates the company competitively that they'd hire the talent. If they're trying to be the best "X" company and nobody who works there is any good at "X", they should fail. The dot-com era is littered with the graves of companies who thought that just having an idea was enough.
Except almost every software startup in the last three decades was started by folks that were not experts in the domain they eventually became successful in.
If my team or I want to maintain it, then build. If not, then buy/rent. I've built way too many things over the last few years only to realize again and again that things we build will need to be maintained by someone.
As you say, if it's a competitive advantage, then it makes sense for me to fix it, update it, adapt it, etc., otherwise it may be too costly to get distracted with maintaining it internally and easier to pay someone else to do that.
Yes. As Joel Spolsky put it in 2001:
“If it’s a core business function — do it yourself, no matter what.”
— Joel Spolsky, In Defense of Not-Invented-Here Syndrome: https://www.joelonsoftware.com/2001/10/14/in-defense-of-not-...
I think we've finally tipped to the point that business processes are designed as digital first. The current gen is also following much more user-centered design.
I always ask them "So are we the only company in the world that is developing software?".
Tools are pretty powerful, so you build your process around them.
Almost all processes work well enough. And there is nothing wrong with trying to find an optimum one. But process is not the solution. Quality management talent is.
You perhaps overlook a few things: (1) Managers are people too. They make mistakes. They should be able to recognize and correct them as others. Ideally quickly, transparently and with the insights of others. (2) Needs evolve. The right process for proof of concept R&D is not the right process for global domination with a mature product. (3) Management is about competing concerns. Often a decision must be made within a limited sphere of experience, time and resources and reviewed in the future. There is no 'right' decision, and it can be fiendishly difficult to quantify decision quality in many such circumstances without the benefit of hindsight. (4) Different people can respond very differently to the same management, so 'bad management' is perhaps too uni-dimensional a notion to hold much meaning.
In this case all I was referencing was that frequent process changes and repeated failures of process are typically an indication that a change needs to happen in management not process. No where did I state management be eliminated altogether. I stated that better managers need to be brought in to replace those that are underperforming.
Tools that get widespread adoption are mostly those that move quickly to add every little feature every manager/exec asks for in order to secure the contract.
Once they become incumbent, they become the standard, because managers are responsible for the 'big purchases'.
There's tons of very accepted systems that are 'not very good'.
The article seems to focus on personal productivity tools, for which the author's conclusion might relatively hold to a fair extent.
For everything else - I agree. The more bespoke you decide to be the more pain you create for very little value.
The military was ordering planes designed for the average pilot. But because there are so many dimensions along which someone can vary, most people are far from the average on some dimension, and when SHF and they need to be able to reach the right thing at just the right time, sometimes they couldn't.
Each pilot is unique, just like every other pilot. Accommodating that proved important.
Now... how well does that translate to building software? I can certainly see reasons why it wouldn't. But I don't see an argument that wouldn't apply to the planes as well, without actually measuring things.
Well, let's dig on on what the Air Force ended up doing, overall.
First, they studied and improved the ergonomics. Specifically, making things like seats adjustable. The tool became more adaptable for a wider range of pilots.
At the same time, there is a limit to how far that can go, especially when we are looking at 1940s technology.
The Air Force thus imposed body-size standards for pilots -- 5'4" to 6'5" is a wide range, but it excludes outliers. Too short, too tall, too wide, or too heavy, and you couldn't be a pilot.
My Dad was in that category -- too big to fly his dream plane.
Controls were adapted so that unique functions had unique hand-feels. Flaps had a different shape of control than landing gear, and all of that was standardized across aircraft.
So what does that tell us about workflows for engineers?
Well, we can say firmly that physical ergonomics matter. Bigly.
Uncomfortable seats, poor-quality monitors, high-latency or slow network connections... all of those are deal-breakers.
But in terms of things like process and practices... standardize what you can, give everybody a sane baseline from which to work, and then empower them to customize as much as possible for their team or group.
At the end of the day, software is shipped by teams.
There's this odd, pervasive myth of the Morlock and Eloi in the software world.
E.g., that engineers are either beautiful artisans in designer specs to which every whim must be catered, or that you just need to shove a pizza under the door every couple of days and production code will then appear on your servers.
Don't open the door, though.
None of that is reality. And to work in a team, you get to surrender some of your personal work-autonomy, in order to leverage the cognitive multiplier effect.
> Well, we can say firmly that physical ergonomics matter. Bigly.
Can we? I don't think that's the right lesson to take from the story. The story clearly establishes that physical ergonomics matter when split-second reactions matter under unusual G forces. I would be astounded if we didn't find that they matter less in the sort of situations most of us find ourselves in while coding. It could still be more than "bigly"; it could be very little. We can pull in other evidence and experience to try and make a determination, but that's no longer drawing lessons from the story.
I think application to software development tools involves asking whether we find, there, too a high dimensional space; and then recognize that, in such a space (where each is picked from independent distributions), we'll find that each of us is similar in a lot of ways, and pretty different in a few. It doesn't seem unreasonable that this would sometimes impact choice of tool.
As you say, teams are a crucial part of the production of software; it's vital that whatever tools are chosen, they mesh well where there is interaction between individuals.
That's for business but I think it applies to individuals as well. You're much more adaptable than software. Pick the thing closest to what you want and change how you work to fit the software.
Some companies have the skill and desire to build their own tools over our products so they can use their pre-existing workflows, but inevitably the abstractions start to leak and they almost always end up hiring us to train them on how to use the tool the way it was meant to be used.
I’m not sure I agree. I think good tools represent the problem space well, and provide powerful ways of manipulating/interacting with it. That may or may not be what people are already doing.
I’d say solving a problem well is orthogonal to what people are actually doing.
More frequently than you could imagine, people and organizations jump to how they are going to implement something before they define what they want to implement. And when the effort fails, the technology is blamed and they start all over again.
They end up finding that garbage in = garbage out
When you adopt a tool you adapt your workflow to its limitations.
on edit: obviously this is an "ideally speaking" type situation.
- When you write a tool without knowing exactly what is a beneficial workflow, and this imperfect, probably under-documented and under-tested tool must be maintained over the long term.
- When you adopt a tool - trusting that its developers know what a beneficial workflow is - and it doesn't suit your workflow, or skews your workflow unnecessarily, where you have to bend over backwards to suit the tool. It can add inefficiencies and cost of time/effort, that might not be worth it - but, once adopted, you could be stuck with it over the long term.
---
The posted article argues for the advantages of "homebrewed tools". The main point being, the priority and emphasis on the workflow itself, which the tools should support.
> I can build the tools that exactly conform to my workflows, rather than constructing my workflows around the tools available to me.
> ..Tools you build yourself can grow and change as your workflow changes over time.
> This is typical of the way I discover my workflows. I start with a minimal, bare-bones solution, and try to pick up on patterns and tricks I create for myself. And then I encode those patterns and tricks into the tools over time.
In my experience, I've found this approach to result in simpler systems that are easier to understand in a sitting. When it goes wrong, it can be adjusted right there in the tool.
And, in general, I feel that when things go wrong with an established, community standard tool, it's more painful to work around it since, usually, it's not easy or possible to change the tool to fit your ideal workflow.
---
There's something to be said for community-developed and supported tools, with detailed documentation and a crowd of people using it in production, essentially testing every nook and cranny of the tool.
I suppose only experience can truly distinguish when one approach is better than the other, in a given context.
½ of the subscription including your time, or no?
I spoke with there legal team and turns out they didn't really need a legally binding signature. New system does everything they actually needed, without any real reoccurring costs.
While hiking around Iceland, for example, one should adopt established tools and methods, and adapt to them. No one's workflow can ever grow to accommodate that level of embedded efficiency and safety.
Real change in business requires adopting new processes. I just helped a small business shift from a paper driven workflow to electronic. Adapting their workflow to fit established tools was less work than doing nothing, less effort than their existing process; they doubled their output literally overnight. We'd have been fools to try to adapt a tool they'd never used or heard of before to look like their paperflow.
In the end every quirky thing you do means more on the job training for all new employees. If you take quirky too far, then you can’t hire new senior talent (everyone starts as a noob). And it may be very uncomfortable to look at, but you need to think hard about who benefits by having things be this way. It’s not the company, and probably not the team. It’s a couple of people and likely also their manager.
In the short term, perhaps, but at what cost in the long-run? We end up with channels of accepted behavior dictated, undemocratically, by software teams.
1. Map these intraprocess workflows by talking to the users, find the biggest pain points for automation.
2. Build highly modular tools that implement those workflows and automate those pain points.
3. Deliver the tools into the environment and wait a bit.
4. Re-interview the users and find places in the workflow that are now the newest biggest pain points. Goto step 1.
It's a pretty simple process and not hard to do. If you do it right, over a very short period of time you deliver working tools that improve the user's day-to-day jobs and then just...continue to make it better.
Fairly often, after a few iterations, you end up just completely automating entire workflows and I've never had users get upset about this. It usually turns out those were boring, dull, and repetitive things they had to do and they hated spending hours of their days doing them anyways. Freeing them up from doing those things means they could move on to higher-value work that was far more interesting for them.
> The Eureka moment that some of us feel when we finally find a notes app or todo system that fits our brains – that epiphany happens when the tools we use mirror the way our minds work, and how we want to move information through our lives. Good tools fit perfectly around our workflows, bad tools don’t.
> When we resort to having other people build tools for us, the tools they build might never quite perfectly fit our workflows, because they’re not built for our individual minds.
He goes on to advocate building yourself exactly what you need.
This is what I've started doing. I needed a site generator, was too lazy to learn an existing one, and built something that would just do what I wanted, and no more.
It's actually quite a nice experience knowing a tool inside and out, literally.
Once I realized that I automate my own processes and I am so much happier. I went so far as to purchase an annual $99 developer license even though I have no intention of selling apps.
However, when considering overall productivity, it’s very easy to fall into the trap of yak shaving, and you’ll end up with doing a lot of work in order to be “more efficient”. This always rubs me the wrong way.
And it is for this reason I typically prefer some integration into a tool that I’m already using, rather than using a vast amount of tools that I need to integrate myself.
In the end I settled on emacs for most of my things here, and since I’ve been using it for over 20 years, the investment into learning ELisp etc pays of a lot. I much rather prefer to work with the framework that eg org-mode provides, than try to find some tool that is “just a little bit better!”
Yes, it may be better, but in the end it will involve a lot of work to tie everything together. I simply don’t have the time to do that.
With ownership a different kind of productivity is evoked: knowing how (and why) every part goes together the way it does. This could lead to a flow state when managed effectively.
I do agree that these exercises could lead into the realm of yak shaving, but in this context we're talking about making personal tools vs tools for others to use. I think there's a big distinction between these two approaches.
Agree on the distinction between building for myself vs. others. There are a hundred things I'd have done differently, were I building for someone else.
The time energy and effort will make the creator more efficient and more effective in nearly every project they undertake going forward in life.
Making and learning, especially if they are in areas related to what you do frequently (for example, as a career) almost always pays off in overall efficiency and productivity.
But when it comes to Software Engineering Workflows, I've learned in my decades of experience that the progenitors of a process or workflow had no idea how to do it, made the assumption that no one had done it before (or that buying a solution would be too expensive), and then just cobbled something together to get the product built and out the door. They start hiring more employees who spend an inordinate amount of time just understanding the existing workflow and soon lapse into "well, it's not my problem - it produces output, and I just need to fix bugs / add features to the product."
Before you know it, you're ten years in, with a couple dozen white-labeling clients, and that build system can't scale.
What I feel like we need is an encyclopedia of workflows. So you wanna make an app: make several decisions now, and here's the recommended design workflow, engineering workflow, testing/staging/release workflows ...
BRB, I think I just found a consulting idea...
Here's a good example. This is a machine making N95 masks.[1] It works about the same way someone would do it by hand. It's the obvious approach. It's painfully slow to watch.
Here's a machine making paper cups.[2] The process looks very strange. It's over 5x faster than the mask machine, and it's doing a more complicated job.
But this guy is talking about phone apps and stuff for people who work at desks, not automating a manufacturing line.
One big problem in the US is that there are not enough smart people making machines like [2]. And too many people making things like what the OP is talking about.
"Some of my apps, like Ligature and Noct, are written in Ink, which is a language I wrote myself"
is both impressive (truly next-level), and also completely out of reach / impractical for the vast majority. But it is inspirational; thanks for sharing -- your other blog posts look worthwhile too...
There are so many great resources for building a toy language by yourself (my impl of Ink is a few thousand lines of pretty simple Go code) and it's never been easier. I really think most people overestimate how intellectually intractable it is. Building a production language? Yes, a lot of work. But a toy language for you to use / learn with? surprisingly doable.
I have a scripts to move clips off of SD cards into a date sorted folder structure. I have three cameras and a phone, and usually want to get clips from the same date on different devices (SD cards) into one folder on my backup drive.
There's also a problem when you film yourself, that lots of footage is wasted without action. Yesterday I recorded a video that was 10 minutes long, but only used the middle 2 minutes. So I used a script to chop up the video, then deleted the useless parts.
I also use Typora (mark down editor) for planning and notes. Highly recommend it.
If anyone is interested:
https://github.com/andrewning/sortphotos
https://github.com/c0decracker/video-splitter/blob/master/ff...
First off, their tools are very beautiful from the screenshots, and if they work for them great! However I can only think of the time liability they are (they even wrote the language they are written in). I can only imagine how much time this person sinks into making them. If that’s what you love doing great! But for the vast majority of people that’s not reasonable. They are pretending that it’s a high productivity approach. It’s not.
Productivity = results / time
Spending a few dollars (equivalent to working for an hour lets say) and getting a high quality tool that would take you months to build yourself, is the high productivity solution. Many tools allow a high degree of customization and plugins, which allows people to really make it their own as well.
Having a super customised calendar, drawing app, todo etc is going to improve productivity by what,10%?
And building all these tools is many months to a year's work?
What if they'd used off the shelf tools and spent their time building something new vs reimplementing so many wheels?
For what it's worth -- it doesn't take me much time to build these tools. Most are an afternoon project / a weekend project. So that also makes this more reasonable for me.
EDIT: I'm also no stranger to off the shelf tools -- I've shopped around my fair share of todo lists and notes apps :) Just haven't found any that worked the way I liked exactly and allowed me to own my data the way I wanted to.
When once upon the time, I decided I was going to write a novel, I spent ages finding exactly the right tool that would do the typesetting exactly right and had all the features I just KNEW I needed. Never ended up writing a damn thing.
Eventually, popped open Notepad and actually did some writing. Felt amazing to just get started.
> one place where my mind works differently than the tools on the market is the task/notes distinction
I wonder if you actually discovered this when testing Notion, which doesn't have that distinction ;)
Part of this is ego I think. Many of us have landed jobs in agencies or at companies who produce commodity software using a framework or tool, and we see things about them that we feel we can do better. But if you're working with a client and they've decided on a framework, part of that decision is the implicit understanding that you will build them a platform that every developer using that framework can work on.
It's not unlike choosing to use Volkswagen vans for a transport business because you have a good community of Volkswagen mechanics. If they ask you for a VW, and you give them a VW that has a Ford engine and a Nissan transmission, they're going to be pretty confused and annoyed when the mechanics tell them they can't work on this and need to rebuild it.
If you don’t mind me asking, how much effort did it take you to develop each tool, and did you find that as you did more tools, subsequent ones took less time to develop and/or were more capable?
These days developing each tool is 50% taking pieces of tools / libraries I've built before and integrating them together. I've implemented a data store, a form UI, a websocket server, etc. each a few times so as I built up a collection of things I can borrow from other past projects, new projects take less time (or, I have more bandwidth to tackle more complicated projects, new a new tool if need be, etc).
I have another blog that goes into my side-project workflows[0] but in gist, I try to make a "usable MVP" for each project within a single chunk of time -- usually a weekend. And from there I try to dogfood it and add changes that I need to keep using it.
All this is helped by the fact that, at least with these tools I use, I try to keep a really simple stack. Go or Node.js server running on a single VPS, light frontend using my own framework [1]
I do agree with your argument about building tools to fit your workflows. Low code and no code tools will bring down the cost of custom tools dramatically.
Companies don't update their tools enough because people hate changing their workflows. I've worked with customer support teams and change in workflows is a big reason why they don't want to buy a 'better' SAAS tool. They'll lose productivity in short-term with a new product that's different.
(shameless plug-I started an open source project to let companies build their own self-hosted tools and workflows. https://github.com/appsmithorg/appsmith)
Very rarely, and if I do I build them in. But my needs are pretty low -- I'm ok with plain markdown/text for storing most data, or nicely formatted tables. I've found that super-rich document formats with embedded this-and-that don't actually add a lot of value for me (but of course this might be Stockholm syndrome speaking, I suppose).
>Does tool maintenance get in the way of your work?
No, and this is a big consideration for anything I build. I try to design / deploy apps so they require minimum maintenance. Static Go binaries + going slim on frontend dependencies both help with this. The only real "maintenance" I have to do is periodically reboot my Linux box and upgrade the Ubuntu version to the next LTS every few years. All my binaries are deployed as auto-restarting systemd services on an small VPS -- not depending on more serverless-style platforms helps keep maintenance needs low too, I find (though there are disadvantages, it doesn't really apply for one dude running a dozen web apps on a single server).
For example: append some markdown text into a given text file; open a text file, tokenize and filter lines where token 1 matches user input; if the user input matches a certain pattern, route it to a given file etc.
I'm already doing this with bash scripts (and they work for me), but for this idea to take hold with non-technical people, we will need GUI utilities.
The most important process I implement with teams is retrospection and its use as a mechanism to modify process in a principled way.
So as you learn about your own process, your tools can either fall away and be replaced, or flex with you and stick around for longer. My guess is that flexibility is slightly better than fixity, as it'll let you live within one ecosystem for a little longer.
The other way of interaction is to be inspired by new tools to modify your process. This feels pretty off to me, at first, in that it means that you're moving your process do to exogenous things as opposed to endogenous learning.
But that sort of work can be really valuable too. It's good to learn other people's methods. It forms an impulse that can get you out of local optima. It's an important part of the learning process.
The moral of the story: Your workflow is not sacred. Even the way your brain "works" is not sacred. Learn to adapt or be prepared to die. If your workflow isn't changing or at least being tweaked from time to time, the odds favor the idea that you have grown stagnant. That's not to say that change for the sake of change is the preferred alternative however. I'm only saying that treating your workflow as sacred and hence immutable is a dangerous trap to fall into.
Tech changes quickly. Not just the technology, but also the markets, customer bases/expectations, design ethos, and delivery/distribution systems.
So does the jargon, but I've learned not to waste too much time trying to keep up with that.
If I get too mired in the workflows, infrastructure and tools, tech can go whizzing past, while I'm polishing my scripts.
In my view your tools should come to you - for instance with CI, why should you have to go all-in porting your production build pipeline for a third-party platform and workflow when you have a build script that already works, and fast, and you really just need to change a couple of flags to build for production rather than dev? As the author says, you shouldn't change your workflow for the tool, the tool should help you with your workflow.
If you already have a workflow, you're already more than familiar with that dance, and there is no advice.
Would you spend effort designing a novel kind of hammer that grips the sides of the nail instead of striking the head? No. Just tell the carpenter to stop doing the stupid thing he has been doing.
The reason Airtable, Notion, Monday, and others are so successful is because they do precisely the opposite of what the author is suggesting.
This is the same reason most workflow software starts out as a replacement to Excel and Spreadsheets, but people continue to resort to Excel and Spreadsheets.
This is great for software developers because it guarantees work, but bad for business because of unnecessary IT costs.
And an other kind of... diversity ?
In front of this screen, polarized, asking myself "Do i became too choosey ?" For what Narcissus -the protagonist is meant, to "think of oneself" is any good for, not ? P-:
I always found that it is easier to make small changes to the workflow based on the tools and the tools will increase throughput and will justify the effort for the change.
* Tiny, you just do things the Quickbooks/Excel/MS Word way
* Gigantic, you just do things the company's way, it can pay for customization
* In between, here you have the major problem
You can do all kinds of crazy thinks with git, but the price is that for many common workflows you have annoying gotchas and or usability annoyances.
But it also seems the author actually enjoys building these tools. So it doesn't actually cost him anything. The work itself is a reward instead of a cost.
I can't say spending 1700 hours making my own tools is what I really want for myself, but I guess different people are different (I'd hate to find out exactly how many hours I've spent editing ~/.emacs though, tbh...)