Yak Shaving: A Short Lesson on Staying Focused (2018)
americanexpress.io
americanexpress.io
On the topic of yak shaving though: I recently felt like I wanted to do more technical writing. The prerequisites of this idea involved building a static site generator and redesigning my website. While building the SSG, I of course imagined all the features that would be expected, a tagging system, JSON output in addition to markdown rendering, custom syntax highlighting for codeblocks etc etc. I had that idea at least 1-2 years ago, and I've only recently written my first post within the past 2 months. I think I enjoy tinkering with build systems much more than writing.
The downside is, it’s not so much the article that becomes relevant, but the topic. HN felt like talking about yak shaving and all we needed to get started (you included!) was a headline.
Do you get different emergent behaviour depending on the time of day/week? Have demographics of the site altered such that the prevailing opinions are vastly different? What is the mean time to only tangentially related items being discussed?
I don't think a culture of 'only reading the comments' is at all a good thing.
*by that I refer to comment chains just making bland puns, joining in threads for no reason other than saying something, even if it adds nothing and at times is straight up incorrect, etc.
For the people that read HN but don’t comment, please load/scan a link before upvoting. Not doing so will have a net negative impact on HN if enough people are not checking the links.
My feeling of things related to yak shaving:
Rabbit holes - intentionally following some thread of associations looking for some insight or unexpected solution
Yak shaving - While working on something important you iteratively find something more important that you feel you need to deal with first. Supposedly but not necessarily it's something you have to deal with to be able to solve the original problem.
I agree that the articles example of yak shaving is a special case of yak shaving where it isn't very clear that the shaver feels that the distraction is more important. But I suppose such feeling is always a bit implied in the word distraction.
No; yak shaving is something you must deal with to continue working on your original "something important". It is solving blockers, not following distractions.
I am at least 18 months into exactly the same hole. And it isn't the first time.
I've been using Obsidian[1] and a few plugins like Incremental Writing[2][3] to help me write. Focused less on publishing and more on writing for me.
[0]: https://www.goodreads.com/book/show/34507927-how-to-take-sma...
[1]: https://obsidian.md/
But even the best of us are prone to fall for it, and sometimes they even produce excellent results, cf. Donald Knuth and the TeX / Metafont ecosystem:
Between 1977 and 1979 computer scientist Donald E. Knuth of Stanford University created the TeX page-formatting language and the Metafont character shape specification language, originally as a way of improving the typography of his own publications. [...] -- https://historyofinformation.com/detail.php?id=3339
Does this really happen?
My experience is that programmers always try to make things as simple as possible, and when they fail at that, it's because they don't know how, or don't have enough time, or maybe the "simple" solution isn't persuasive enough that it covers all the cases, so a "more complex" version is preferred, or something like that; not because they forgot "whoops" I was supposed to make things simple not complicated.
The only times I have heard "KISS" suggested in earnest have been from bad managers.
> But even the best of us are prone to fall for it, and sometimes they even produce excellent results, cf. Donald Knuth and the TeX / Metafont ecosystem:
I don't see how this has anything to do with failing to keeping things simple, yak shaving for sure, but this yak (if it's a yak) did need the shave: there weren't any typesetting systems comparable to TeX at the time (and by some opinions there still aren't). What else can you do? Have someone spend hours typesetting and resetting a thousand page book every time there's an error, or turn the job over to a computer and automatically typeset it?
This is just solving problems. Solving problems you have is good, and where people go wrong, it's when they are solving problems they don't have (but perhaps think they might), or perhaps they underestimate how much of their problem the computer could actually solve.
This is very much an easy trap to fall into! What helped me was not sweating over the small stuff and setting up an instance of Grav, though I think that most of the turnkey blogging solutions out there would work (e.g. Ghost/Bolt as well, maybe not self-hosted Wordpress as a first option due to large surface area): https://getgrav.org/
What I really like about that solution in particular is that it is flat-file based and also has an admin web dashboard that's a separate plugin that can be enabled/disabled (some might prefer writing text files directly, with front matter and all) and has separate URL path that can be put behind basicauth (in addition to built in auth), client certificate auth or anything else.
It's not perfect, of course, and has given me the occasional headaches, but it's also good enough for my blog: https://blog.kronis.dev/
That said, I still struggle with my homepage - instead of going back through 5+ years of projects and describing all of the noteworthy ones, putting up a few galleries of screenshots, listing technologies, ordering them by relevance and also making sure that it doesn't contain too much data... it's just sitting there, on my TODO list. It's been that way for a while now.
I want it done. But I don't want to do it.
To clarify: that is not yak shaving, that is just plain old procrastination. Yak shaving includes tasks that you have to complete in order to do whatever your original intent was. For example:
* I want to add new feature to my code.
* The feature requires a database migration.
* To apply the migration to test environment I need to set up AWS credentials.
* To set up credentials I need to log in to AWS console.
* My password hasn't been rotated in a while so I need to update it.
* etc
The example (and definition) in the actual article disagree with you:
"You want to bake an apple pie, so you head to the kitchen. In the hallway, you notice some paint chipping on the wall. So you walk to the hardware store for some paint."
You obviously don't need to deal with the paint chipping before baking an apple pie.
Wikitionary gives both definitions: a blocking task or a procrastination detour
Create a GitHub repository and create a journal repository and create a markdown heading for each entry. It works.
I'm on 4 pages of 100+ journal entries each. See my profile, I journal computer ideas, algorithms in the open.
Using WordPress or using complicated software shall take time away from writing regularly. And you need to open a document and write, not abandon your blog due to forgetting how to post
When quasi-blogspam articles like this rise to the top so quickly, one has to wonder whether it’s the result of vote manipulation.
Uniqueness, mystery, narratives and humor/silliness are probably big drivers of attention on forums and news sites. Like the other commenter suggested, the gold is typically in the comments. I don’t have data to back up this assertion, but it would make for a good study/article.
The magic is in the Ren & Stimpy reference.
So now instead of fixing my bug I’m rewriting API call sites across my project.
I quite like the Malcolm in the Middle[0] example of yak-shaving, though it's a combination of both blockers and distraction.
> yak shaving - Any seemingly pointless activity which is actually necessary to solve a problem which solves a problem which, several levels of recursion later, solves the real problem you're working on.
I guess that opens the debate to does the code actually need to be refactored? Sometimes not, but I have a very hard time leaving a 2000 line function alone, or a class with many levels of unnecessary inheritance.
What I’ve started to think is that there are people who get things done, and people who refactor.
People who get things done ship code fast. The good ones know when to make trade offs of purity vs practicality. The bad ones write unmaintainable code. They tend to view programming as a job, and code quality as a burden. They may refactor and make unrelated improvements, but it’s uncommon.
People who refactor still get things done, but most of their work have rabbit holes tacked on. They delivered a bug fix, but they also greatly improved the test suite and tooling. They tend to value purity, best practices, and might view programming as a craft/art. The good refactorers make the high value changes that are a net benefit to the team; they might care about cleaning up important parts of the code, improving developer experience, or adding test coverage to critical functions. The bad refactorers rewrite a bunch of code because it wasn’t pure, and the team, customers, and business have no observable effect from it.
I think teams need both kinds of people on it. Too many doers will lead to fast progress at first, but eventually grind to a haunt. I felt this was a HUGE problem at my team in AWS, and a major factor of why I left.
Too many refactorers is equally problematic. Your code will be of great quality, but velocity will be unacceptable for the business.
I personally am a refactorer. probably more of a bad refactorer than good, but it’s something I’m working on.
Maybe this is a rationalization for my behavior so that I believe it’s important, but my time at AWS shipping 3 features a year with a team of 16 says that code quality really does matter.
It's a matter of focus and intentionality.
'Phase 2 never happens'
Yes, that's probably a sign of poor project management or organisational brokenness. But it's also probably the reality for most people.
> It also misses the point that sometimes refactoring makes it _easier_ to fix the bug, and that a large part of fixing a bug is understanding exactly what's happening with the code, which refactoring can also make easier.
I clicked on this article because it applies to me (I constantly yak shave), but I’m a bit disappointed with how terse it was.
I have had this conversation yesterday, and the refactorer asked this very question. Here's my answer:
The question is improperly formulated. With your behavioral patterns of immediate refactoring, the question seems to be: "if I don't refactor this code NOW, who will?".
I'd say that if you don't refactor the code now, someone else, maybe future you, may refactor it later.
Most sane companies will protect engineering from impulsive feature requests and will add them to the backlog for future consideration.
I'm trying to instill the same protective mindset in my teams, and help them not do refactorings immediately but rather consider them as potential work as dedicated items in the sprints or so.
Doing so, prioritizing becomes a natural part of the refactoring/tech debt/tech risks agenda, because we have an overview of the work to do globally.
I guess this is delving into queue theory and how the work queue is growing to infinity because there aren’t enough workers.
I miss personal projects
I’m still trying to internalize this hah
Meanwhile, I can't get refactor-only commits through because nobody wants the code to be touched unless absolutely necessary.
We're in the middle of migrating a massive PHP codebase to Java. The code I'm talking about is newer Java code. In a few years, people will be wondering why the Java code is as much as a mess as the PHP code and, if I happen to still be around, my response will effectively be "I told you so" (in kinder, more productive terms).
I've basically determined that my only option to ensure continued or improving code quality is to combine refactoring and bug fixes in the same merge request, while still keeping them in separate commits. That way whenever I touch a part of code for bug fixing or whatever else, I leave it behind in much better state than it was before.
I agree, in practice I think this is the best option, with some kind of timebox for the improvements made.
One thing I'd like to add is setting up some kinds of tests. If nothing else is available, E2E tests for the critical paths of the business are a good start. For (micro)services, getting API contract tests to work is also invaluable.
The guy the goes around blasting bugs like there's no tomorrow looks good to the boss. When he leaves someone will have to refactor his mess and that's wasting time again.
It’s about “necessary” things to achieve a goal, without appropriately weighing costs vs benefits nor considering alternative solutions (perhaps making a chocolate pie instead).
In this scenario you are the one focused, the non-technical people that shuffle tasks around playing 'agile' are the handicap. It could take a couple of hours or a couple of days to get that logging done but explaining to the non technical manager (who refuses to learn) why this needs to be done is a task in itself that cannot be underestimated.
I’m sorry, but this is backwards. Bugs are many times caused by badly written code and you can tell when it’s the case. Refactoring the code many times fixes the issue without ever having to figure out where the needle was in the haystack.
I guess in every field there are platitudes and prescriptions. At the end of the day, I try to follow first principles and ignore them and just focus on building a great product.
What I see in this article is a disregard for the costs associated with context switching. My argument is, if you think you can handle the rabbit hole, and you think those related tasks will need to be done anyway at some point, head off to Wonderland. Because you have the context of the situation fresh in your temporary memory, so you’ll get it done faster than if you switch contexts and come back later.
Overall I disagree with this article, both in it's definition of yak shaving (as being unrelated to what you're doing), and in it's assertion to never refactor and fix a bug in the same PR (now I'm not saying you should refactor things every time you fix a bug of course).
With this kind of framing we can look at the cause of yak shaving as a failing of an individual, rather than a lack of an accurate and holistic view of what the task entails given the current state of the system.
My point however is not to discuss this. What bothers me, and only a little, is the label chosen to describe situations such as these. Why 'yak shaving' and not something else?
I mean, the Ren & Stimpy connection is meaningful for the author, and that's ok. But if every time I want to refer to it I have to explain why it's named as such, or even explain who Ren & Stimpy are, maybe I'll be better served by some other, perhaps more appropriate label.
Calling it 'Going down the rabbit hole' is no doubt better, at least to me, but not completely. This expression captures a different mood, namely that of finding ever more connections in an endless stream that does not necessarily make me forget my original goal. For instance, while writing a paper, many times I find myself going down the rabbit hole, ending up with these monstrous footnotes that try to record how deep that rabbit hole goes. But when that is done, I go back to the main topic and move on.
So what could I call it other than 'yak shaving' or 'going down the rabbit hole'? I'm open to suggestions.
Nothing else about the hell that one in is in when they are yak shaving is the least bit funny. At least give us this little bit of joy.
It may be more recognisable, but I wouldn’t say it’s better. The metaphor of going down the rabbit hole has you go deeper into the same subject while yak shaving has you go into increasingly tangential tasks to the point an external observer wouldn’t recognise your primary goal.
In other words: the longer you spend going into the rabbit hole, the more you familiarise yourself with the matter; but the longer you spend yak shaving, the farther away you are from finishing the original task. Yak shaving is antithetical to going down the rabbit hole.
This person has obviously never been in a situation where to get the data to fix the bug you need to modify the instrumentation, which requires a new data format because you are the first to use dicts of dicts of floats, and when adding that you end up with a random seg fault, so you debug that and find out that the stack frame pointers disappeared last week when GCC was upgraded by a vendor patch on your colleague's laptop, but not yours, and so he checked in a "temporary" fix for that. Frame pointers restored, you find an off by one in the serialization of floats which you fix. So you go to collect your data but the data aggregation system, which said it supports your data type in fact does not in the current version, so you upgrade it, but for that you need a different GCC update. But that GCC needs a different libstdc++ which causes compile errors in your app. So you find yourself in the Makefile adding a way to use a local copy of libstdc++, and you can't remember why you were even fixing the bug, because in the meantime the docker image used for the mock database has been updated, adding a new bug that masks the bug you were supposed to be solving (but you won't find out about that for 3 more months).
Your scrum update for 6 days running is "maybe tomorrow", and your cat runs away because you are so preoccupied with the bug you forgot to feed it, and your yak needs shaving because in a daze on the way home from another day wasting your life at this stupid job, you forgot to pick up razors.
I think in programming it is easy to get lost in scale. We all know these videos where you scale in from the universe into a single atom. In programming tou operate at similar dieseperate scales. You need to have the high level perspective ("I want to get to this mountaintop!"), the mid level perspective ("I chose to scale the mountain via that route") and the detail level perspective ("I put that finger into that nook of the rock to my left and shift my weight slightly to the right..") and of course there is also a tooling perspective ("What kind of gear do I need to scale that mountain?").
When you operate just at the detail level it can happen that you end up on the wrong mountain, branch off into the wrong route etc.
To make things worse you can also have a meta-perspective ("How will this help me beyond the mountain I am currently on?").
distractions are real, at some point I pomodoro technique helps, however when I get interested/distracted with something else at the code then it's pointless
This brings me to my next point: know what you're supposed to be working on. I like to write it down at the start of the day while I have breakfast, before I touch a computer.
Somehow, in practice, I've seen the fixing plus enhancements work better in getting the code in a better shape.
Also, why are we not good with doing both at the same time? If its because it makes things harder to review, I've found that pairing on the review and adding clarifying comments to the PR is a much better way. If its for the commit history, is it always more important to have a squeaky clean commit history than to have easier to read (I'm presuming this is the intent of the refactor) code? Is there some point after which the code is bad enough to sacrifice some commit history clarity to get better code?
> You may even decide that these enhancements are distracting you from your immediate goal of shipping a feature. In this case, there is nothing wrong with addressing them in a tech debt sprint.
Has anyone actually seen this work? In my experience, once the original bug is out of the way there is no motivation or vision on how or why to make the cleanup and the tech debt tasks are left to rot forever, while the code slowly turns into mud. Does anyone know how to avoid this?
IMO the thing that works best is to do the yak shaving first before the bugfix. But also, timebox the effort for yak shaving - you're only allowed to do a certain amount.
- You have the best mental model about a piece of code and its system the best when you are working on it, not when you read a ticket description of what should happen. This leads to overall better efficiency and less context switching.
- I think that the principles of "leave things better than you first found them" and "with every not-so-minor change, one should think about how would you architect the system the best way possible at this current time" great principles.
- I find that "Refactoring sprints" never really work great. They tend to be inefficient and rarely prioritized tasks by the management. As developers we have responsibility over code and make sure that its in the best state possible as an implicit description of our profession.
Even though it pays to be focused, I think there is merit in the exploration of code-base as it helps figure out new ideas and avenues of improvement.
[1] https://www.conventionalcommits.org/en/v1.0.0/ [2] https://github.com/RefactoringCombos/ArlosCommitNotation
We’re kinda going the opposite way: If you find something that should be fixed, then fix it.
This is, of course, meant for small things, like renaming or rewriting expressions to make them more readable. But we have had to make this change, because our experience is this is necessary to avoid code rot.
There are a few benefits
If your refactor is large, you can get a bug fix in ASAP in a small PR, and get the large PR reviewed later. Sometimes I’ve done a refactor with a stubborn CI integration test or two that might take a few days to fix due to long feedback cycles.
Aside from that it lets your reviewer enter a different mindset for each change. The bug fix is more easily understood, and the reviewer expects a change in behavior. For a refactor the reviewer can usually review in a more relaxed manner because they know the behavior can/should be the same, and good unit tests should identify any differences in behavior
https://news.ycombinator.com/item?id=32746200
Oh well, back to yak having under the bike shed.
If I’m implementing a new feature, should I also disregard the need for refactoring?
A more nuanced approach is needed. You need to learn when to make changes additively and when to reshape the code to fit your new use case (and how much reshaping is required).
As an aside: I think tech debt sprints (if needed regularly) are often a sign that you aren’t developing software sustainably day to day.
If the code is a mess and I'm going to refactor it anyway should I spend a lot of time trying to fix the bug in the old code only to refactor it afterwards? Or should I refactor and deliberately leave the bug in there only to fix it afterwards?
To me doing them together often makes sense and I don't remember code reviewers ever complaining. But maybe I'm missing something?
- You can deliver the two individual parts quicker
To quote Kernighan and Plauger: "Don't patch bad code: rewrite it".
* It can make the bug fix diff much easier to read/understand
* It makes the refactor a dependency of the bug fix, so it doesn't end up getting dropped as "not important enough"
Of course, if the refactor automatically makes the bug go away, then it's fine to do a single PR, because the refactor is the bug fix.
Small refactorings and cleaning alongside bugfix are fine. Big refactoring should happen independently, of course.
Good code is code that works and solves a problem right now. Otherwise, we will not have clients waiting for a "well-polished" formatted product that is easy to read and maintain for us.
And that is the problem of the whole infrastructure. We needed solutions for X, Y, and Z just yesterday. And then, if it "shoot," we like idiots yearly fixing all problems that have been done earlier.
Speed is critical. More important than perfectionism or wish for well-structured, easy-to-read, maintained code.
Is that awful? Yes. I want my code to be good as it can be. But is it possible to do it in a limited time when an urgent problem must be addressed? Nope.
And over time, we increase complexity, multiply complexity, and then leave because not able to maintain it anymore. That's why we touch different things while fixing everyday problems because we do not want to fix them later. Balancing complexity in future. But this is fake. This future never happen in 99% cases.
The biggest problem is that if we do not rush right now, there will be no tomorrow for the product we develop.
The thing is, poorly-structured, hard-to-read, unmaintained code is the enemy of speed. I don't think it's ever a good idea to go too far in either direction. If you spend all of your time on refactoring/cleaning up code, you never get anywhere in terms of functionality. But if you never refactor, you're development speed slows to a crawl, and making meaningful changes becomes impractical.
Sorry for debating with you. But any modern code editor has tools which solves the problem of "well-structured" code. What about "easy-to-read", this is depends on language, and the programmer.
I've seen so damn much beautiful code, with 1 character long vars. And such code extremely hard to understand. The code is compact and beautiful. But for understanding the code - required a lot of time.
I think, and my own experience proof that. If you not in rush for finishing things just in time while the things actual - you lose. Always. And there will be no second or third chance.
That's why need to write how you used to write. Only experience & practice give you ability to write good code.
Focusing on re-writting some code parts while fixing a bug - big problem. If this re-writting thing does not change anything instead of "better understanding and better ability to read" - this is bad time wasting.
If you not in rush in developing things and doing right actions in right time while the problem or request actual - you lose. Always. Without second chance.
That's why need compromise and write shitty code for winning competition in short run. And then, when you will have audience for your product, you can always fix here, and change there. Nobody will complain about bugs or issues until it leak personal data.
My coworker copy pasted a bunch of stuff into each page, so when I actually templated them I could fix the bug he had in every page, not just every time I find the bug
The problem with "quick" fixes is that they often end up being slower, because hacking a fix into a complex code base often creates new bugs/problems, and you end up rolling back your broken "fix".
> Only experience & practice give you ability to write good code.
I agree, which means that always writing shitty code to get it out the door means that you will only practice writing shitty code.
There's a balance to be found.
But if you need to fix it now: stop what you're doing and _roll back your changes_. This forces you to keep your changes small and self contained. There are other steps involved, such as writing down your steps to build a dependency graph, but the simple act of rolling back your changes is quite powerful. Don't want to throw everything away? Then maybe the thing you're trying to fix is not that urgent, and you can deal with it later.
[0] https://mikadomethod.info/
Tangentially related: I wish there was a way to record my code changes as executable actions on the AST, and attach it as metadata to a commit. As a simple example: I want to change the signature of a method, rearrange the parameters and add a new one. All you get from source control is some text changes, without an understanding of what those changes mean. If you need to reapply those changes you're stuck with the diff tools of a text editor. But if my IDE could understand those changes, it would be trivially easy to redo the refactoring actions. It might not be automatic (what value should this new parameter have if there is no default?), but at least you have some higher level tools available for dealing with that. To take this further: the changes might not even be in your codebase. A library was updated with breaking changes? Just execute the changes you can, and prompt for action when the changes fail.
I feel that recording your changes as actions on the AST is a powerful concept that needs further exploring. Sure, some things don't make sense to record on a higher level. If I add a new method it's much more readable to just add the code as text. But for anything remote complex it would be great if I can express my intentions as executable actions instead of (or in addition to) textual changes. Find all calls to functionA where parameterX is-a typeY and parameterY.value2 is not null, and make sure the caller sends the result of functionA to functionB in a new transaction. Kind of like how you can record steps with Selenium IDE and use that to create something with clean api calls. And the best part: because you've already made the change it's easy to test if your higher level change results in the same change in your source code.
>In this case, there is nothing wrong with addressing them in a tech debt sprint.
Let me guess how often these happen... Maybe once a year to.. never?