Book Review: "Tidy First?" By Kent Beck
pathsensitive.com
pathsensitive.com
Intriguing. I assume the author would prefer a stronger-typed alternative like explicit parameters and enums. Yet I wonder if having a small DSL-like syntax is actually the better for a scripting language. Most of these plots will be hacked together in a local notebook anyway.
What would be a better alternative to this terse DSL in such case?
Later, Python's matplotlib library started life as an emulator for MATLAB graphics in Python so naturally included the same plotting DSL.
Only later still did matplotlib morph into the defacto general Python plotting library. And then because plotting is so complex and matplotlib exposes so much control, most subsequent plotting libraries were based on matplotlib, opting to add value via high-level abstractions and better defaults, often exposing the underlying matplotlib objects to allow for fine-tweaking. And so the linestyle API leaks into those libraries too.
All of that to say, this DSL was likely invented by a scientist in a lab in 1981 and has survived through inertia and "jumping hosts" a couple times, rather than careful design.
I think the DSL is bad. And the matplotlib developers may agree, because while you can pass a combo like "ro--", you can also pass these parameters separately and more descriptively like
color='#f00', linestyle='dashed', marker=matplotlib.markers.CARETDOWNBASE
What's worse, using a terse linespec syntax and being stuck in Matlab-world, or using a terse linespec syntax in Python, and at least possibly being exposed to relatively modern and maintainable software practices?
My the most favorite fragment of this review:
"But then he [Kent Beck] spends 2/3 of the book talking about how to schedule time for tidying [...] code"
"And then when I asked him about this, he actually said that I’m right but should wait for his next book [...]"
Yuck. At least this Kent Beck guy is honest.
Still, I tend to do this kind of work on days when I’m not feeling great. I work on a large, reasonably old codebase (28 years old) so tidying busywork sometimes leads me to someplace interesting.
We’re actually still using Subversion for our main codebase and mostly happy with it. Being able to use --ignore-revs-file is a reason we might want to switch some day.
This has pros and cons. Let's explore them after an example.
Scenario: Big & Noisy PR
We have a PR with 13 commits : 3 fix commits, 5 refactors, 2 doc updates, a CI modification, 1 feature commit, 1 test update
The commits are in random order, and some of the commits are revisions to earlier commits. It's hard to understand the narrative of the commits and review things accurately.
Solution Part 1: Tidy the PR
1. Reorder the commits by type and relevance: 3 fix -> CI ->5 refactor -> 2 doc -> test -> feature.
2. Squash logically similar commits: fix -> CI -> 2 refactor -> doc -> test -> feature.
A squashed commit should have a bullet list detailing each changed module and scope of change:
"""
Fix(package a): Fix a, b, c
This patch fixes:
* (module b): fix [...]
* (module c): fix [...]
* (module c): fix [...]
"""
3. Review and CI the PR
4. Add commits needed to complete the PR
The tidied PR was easy to understand: We fixed some pre-existing issues, beefed up the ci, then set the stage for the feature. The feature itself was clear and simple.
If you do just this, your history will be much cleaner.
Now I'm going to recommend something controversial:
Solution Part 2: Squash-rebase the PR onto main
Yup. Take all that work and mash it together. The final commit message should look clean and detailed:
"""
Big Shiny Feature
This patch implements [...]
Feat:
* (module d): Implement big shiny feature
Doc:
* (readme): update feature list
* (userguide): add tutorial for feature
CI:
* (workflow a): modify [...]
Fixes:
* (module b): fix [...]
* (module c): fix [...]
* (module c): fix [...]
Refactor:
* (module a): rename [...]
* (module b): delete unused [...]
[...]
"""
Benefits: - We drop 7x fewer commits onto Main - Project history is more legible - Commit messages are detailed and useful - Bisecting takes log 7 = 2.8 fewer steps
Risks: - File diffs can be illegible if feature work intersects with refactor or fix work - There are 7x more defects per commit - It is harder to uncover root cause if we bisect
Conclussion
Tidying PRs before review is a no brainer - it greatly improves our review and history.
Squashing PRs onto main loses some information, but can make history easier to navigate. Since we're disciplined and detailed in our commit messages, this is often much less of a footgun than it might seem. Each commit is now a logical and self-contained unit.
Used well, I think it should help with this problem.
git config --local blame.ignoreRevsFile .git-blame-ignore-revsThat's an interesting take. I've usually encouraged removing commented code because every line of code is a liability, even if commented out.
With that being said, commented code is rarely useful and it cna be a liability indeed.
Maybe put a comment about the removal if you want something that can easily lead back to the commit which did it.
Still, commented-out code is generally worthless and should be deleted. Unless it is actively being worked on and only commented out to achieve some short-term goal. In which case it should also not be commented out but instead live on a branch.
Occasionally you might have two flows and you're not sure which is best. But in that case keep them in separate functions and one of those will not be used but can still be covered with tests. And all static analysis tools will work on it. Just needs a comment about why this unused code is there.
- gave a detailed overview of the book
- gave an actual opinion, instead of just the summary
- included specific excerpts to support the opinion
- [bonus] talked to the author about specific questions the reviewer had
And I appreciated this last paragraph:
> So, if you’re considering buying this book, just purchase a subscription to his Substack instead. He’ll earn more and you’ll learn more. The only one who loses is the publisher.
(Link goes to: https://mirdin.com/downloads/notes-on-tidy-first/)
Follow the link and you can buy the detailed criticism of the book for $25. Which is more than the cost of the book new on Amazon ($20, or $24.99 for the Kindle version). Seems somewhere between scummy and scammy.
Disclaimer: I do really enjoy this book, as it reminds my of uncle Bob’s Clean Code, but a short version, and with the focus on what to do after writing code, when you need to change it or want to understand it better.
Yes, it is structured very much like a collection of blog posts. Which is great for me, as I typically work on learning one “tidying” a day or less. So I am not yet done with the book, end-to-end.
I don't see the scummy or the scammy part here. Am I missing something?
There's nothing deceptive or sneaky about it. There's value in distilling information from a bloated book down to its useful ideas.
There are lots of books that are just existing ideas repackaged for a different audience. For example, Atomic Habits is basically a repackaged version of BJ Fogg's research papers. And people see value in that, so why isn't it okay to do that in a more 1:1 way?
> There's value in distilling information from a bloated book down to its useful ideas.
Yes, but it's not clear at all that this $25 limited-time-only set of notes actually does that, and they cost more than a book that is decidedly not bloated (it's 100 pages, 33 chapters, and each chapter is quite short).
I'm less convinced by the criticism about the notes potentially not being worth the money. That's true of basically all products.
In this case, if the buyer chooses to buy these notes based on the minimal information the author has shared, then the buyer should be ready to accept the possibility that the notes won't be what they expected.
If you look at the link, that's what he actually says.
Sometimes these notes are not things I want to share with the whole world, but might be helpful for a few people.
So what do I do? I release them, but only for a limited time. I also put on a price tag — not because I expect to make any money, but so that only those genuinely interested will read it. The cost is $130.00 from ANSI or $162.50 from Global. Copies of
the original X3.159 (including the Rationale) are still
available at $205.00 from ANSI or $200.50 from Global. Note
that ANSI derives revenues to support its operations from the
sale of printed standards, so electronic copies are _not_
available.
The mistitled _Annotated ANSI C Standard_, with annotations by
Herbert Schildt, contains all but a few pages of ISO 9899; it is
published by Osborne/McGraw-Hill, ISBN 0-07-881952-0, and sells
in the U.S. for approximately $40. (It has been suggested that
the price differential between this work and the official
standard reflects the value of the annotations.)When something goes well the anonymity that “the team” creates to the wider business means that praise goes to the project managers and product owners who, realistically, probably did very little beyond the kick off to deliver.
Agile is a system that creates misery for devs.
In the 20 years since, agile morphed into the agile industrial complex, where practices at best cargo cult the original idea.
You might make the case that Kent Back has actually put together two lines of code and run a compiler (even started JUnit) and the thousands of imitators who talk just like Kent Beck haven't, but if you're going to be critical of the agile-industrial complex you have to be critical of the founder too, who created a toxic style of discourse and a piratical business model of agile consulting.
Go back to 2001. Which non-agile methodology would you have liked instead? RUP? How about we all end up in the PMBOK circa 2001?
We take releasing monthly, weekly, or faster for granted. We take continuous integration for granted. So many of these ideas came explicitly out of agile delivery or were popularized because of it.
Granted to do sprints you have to have a better build process than a lot of shops had in the 1990s but taking a week or two to deliver just because the process says so... means a 1-day delay can snowball into a delay of weeks or months.
Also there are all the meaningless meetings that people dread. The standup that inexplicably happens first thing in the morning when you struggle to remember what you did the day before. The retrospective that I only want to answer with "I am so tired at the end of this sprint that I just want to go ride my bike in the hills or drink some beer or smoke some pot or play a videogame, not sit around uncomfortably in a group of people that are either disengaged and hiding it or doing a good job of pretending to be engaged"
---
To me it is fighting works to bring up the PMBOK because the PMBOK doesn't specify a particular process but it does enumerate the things that have to be managed to manage a project, in "agile the good parts" you are just addressing all of these on a weekly cycle instead of a yearly or longer cycle. I also see the rejection of PERT charts and other dependency managements as a fatal flaw for A.I. and data science projects where model training might take 1/2 of the sprint so if you don't start building your model in the first half you blow. your. sprint. every. time. I first saw people make this mistake 12 years ago and they are still making it. By making people pay attention to a bunch of fake management metrics (story points) you distract them from paying attention to the metrics that matter for a particular app. I've even seen a lack of attention to dependencies be quite harmful to teamwork in more normal software projects because if a team really understood that getting Task A done means you can be efficient at Task B and Task C they might get as much real work done in one sprint than they wind up getting done in three.
Agile in the workplace is not for you, it's for management.
You can micromanage in any process.
Agile is just scrum with less steps.
That said, the idea above that you can do software in year long release cycles is pretty insane.
Release cycles and amount of stakeholder feedback that should be involved in the development process is unique per business and industry needs.
I have worked in both (albeit as a junior dev), and the culture of the environment and the severity of the deadline meant much more than the space between deadlines. Deadlines at a tax company are always going to be worse than one with no high and low periods. Missing a back to school launch in edtech is going to be catastrophic, and I'd rather know sooner than later so that we can pull things out of the release.
In the massive integration days, everyone had already done most of the work and there were a lot of pieces that were 70% complete, and the integration dependencies made it brutally difficult to pull something out of a release. I will defend short iterations all day long.
Remember that "sustainable pace" also came out of the agile community. Kent Beck called it a "40 hour week" in XP.
> The standup that inexplicably happens first thing in the morning when you struggle to remember what you did the day before.
Does nobody else keep notes on their days? That said, what is important at a standup is "who needs help" or "who is waiting on something?" e.g. what is falling through the cracks? It helps to get this information on a daily basis, yes.
> The retrospective that I only want to answer with "I am so tired at the end of this sprint that I just want to go ride my bike in the hills or drink some beer or smoke some pot or play a videogame, not sit around uncomfortably in a group of people that are either disengaged and hiding it or doing a good job of pretending to be engaged"
I will defend retros, but only if they result in changes that make the team a better place. If I was in a retro that had low engagement, I'd explicitly call it out in the retro that the format is not generating change.
> To me it is fighting works to bring up the PMBOK because the PMBOK doesn't specify a particular process but it does enumerate the things that have to be managed to manage a project, in "agile the good parts" you are just addressing all of these on a weekly cycle instead of a yearly or longer cycle.
I'm not talking about 7th edition PMBOK here. Let's go back to the 2000 edition.
"Each project phase is marked by completion of one or more deliverables. A deliverable is a tangible, verifiable work product such as a feasibility study, a detail design, or a working prototype. The deliverables, and hence the phases, are part of a generally sequential logic designed to ensure proper definition of the product of the project."
2.1 and 2.2 talk about waterfall and spiral before heading into stakeholder management. It follows through the entire system - consider what integrated change control looked like in those days. It's all things we take for granted now - when was the last time you had to write a design doc that was longer than 2 pages? When was the last time you got in a room to hash out and negotiate scope for a year of work? It's truly brutal!
Now, this isn't to say I love Scrum. I agree that if you are letting your metrics get in the way of your work then you've done the exact wrong thing. If you know you need to start building your model early then you start building your model early. If you aren't paying attention to your dependencies, then the process has failed and maybe you jump to something else.
I'm not even saying there are no flaws in agile, Scrum or otherwise! However, so many of the things that came out of agile were net positive.
That's a very strong statement for which I don't believe you have the data to back it up. He (or his practices) has not 'destroyed' software developers. Not sure if you were working in the pre-agile era, I do agree some shops take to the extreme, but what we had before was a complete mess.