I Have a Dirty Secret. I’m a Software Craftsman
levelup.gitconnected.com
levelup.gitconnected.com
What strikes me with all these articles is that good code is never objectively defined. It's always some arbitrary measure, sprinkled with common wisdom (write tests! Keep functions small!). The author assumes that they write GOOD code and those evil managers and business people actively conspire against them for writing GOOD code.
I suspect most people who write these articles aren't as good as they think. Most genuinely good engineers know the value of code, and how to balance their objectives with time. Being difficult to work with by insisting on arbitrary measures (e.g. all functions need to be pure and follow my naming convention!) does not make one a good engineer.
There's a reason good communicators advance in their careers while grumpy tech-leads who constantly bring problems stay at their level.
It's funny, on this site I've been chided several times when I talk about iterating on quality. People are under incredible pressure to make deadlines, and they externalize that with a "if it works once, ship it and never look back until it breaks" attitude. Is it an employer conspiracy against quality? No, that would be ridiculous. Is a derisive attitude towards quality an immediate and obvious consequence of focusing on tangible and immediate delivery above all else? Yes, of course it is.
What does iterating on quality even mean? I challenge you to define this term in a way that is clear and objective. I guess the whole point that I'm trying to make is that "quality" is never well-defined, and thus is basically an arbitrary goalpost used by mediocre engineers to throw a wrench in things and justify their existence.
Thanks, but no thanks. It's highly context dependent. Performance can be a goal. Simplicity (rather, lack of useless complexity) can be a goal. Maintainability can be a goal. Generality can be a goal. Understandability can be a goal. There's a sweet spot where brevity can be a benefit to those goals, beyond which it turns to tortuous code-golf. Taken to an extreme, any of the goals above can become pathological. With experience, you should know the difference between shoddy, slapdash or tortuous code and well-written code.
> With experience, you should know the difference between shoddy, slapdash or tortuous code and well-written code
You've just successfully proven my point. By refusing to clearly define what "quality" means, you can now arbitrarily draw the line for quality anywhere you want any time you want. Any time you feel like you're not feeling important enough, you can throw a wrench into the process, yell about "quality," then complain on your blog when your manager/higher-up inevitably overrides you. If you ultimately want to provide value with your work, this isn't how you do it.
> Any time you feel like you're not feeling important enough, you can throw a wrench into the process, yell about "quality," then complain on your blog when your manager/higher-up inevitably overrides you.
You don't know me, then. It sounds like you work with petty-minded hobgoblins. I apologize for their existence. And for the record, I don't feel important and I generally distrust people who strive to.
> If you ultimately want to provide value with your work, this isn't how you do it.
I agree, posting to HN is strictly value-negative.
In my experience most genuinely good engineers almost never complain, either.
The actions of Linus will have ripple effects on millions, and he doesn't have too many levers to pull. Being grumpy is just one of his levers to pull against engineers backed by billions. Compare this to a grumpy tech lead, who's actively throwing a wrench in things by insisting on a nebulous "quality" just to justify their own existence. One is just a pale imitation of the other.
Obviously, Linus has promised to reign in his grumpiness, and I'm not smart enough to even sniff his wrath, so I probably don't have perspective.
This guys code might not be as readable as he thinks. I do not want to read through your hundreds of 10 line functions because a few people said "long functions bad".
I liked the portion on naming. My God the amount of AbstractFactorySingletonViewModel garbage I see is astounding.
I don't impose fixed line counts for my methods. But when it makes sense, the readability improves a lot.
let foo = Foo::new()
.with_bar()
.with_baz(bak, bat)
.listen();
for f in in foo.iter() {
if f == 1 || f > 9 || f % 2 == 0 || is_edge_case(f) {
ret.push((f * 100).to_string());
}
}
Single responsibility is an engineering principle that is not limited to software development. It is the reason that we don't combine the breaking functionality in our car with the am/fm radio. We keep moving parts isolated because the fewer "things" that something does, the less likely it is to break. The less complicated it is.
But there are other side effects to creating small, single-purpose functions (or anything for that matter). It is not always obvious when you will have an opportunity to reuse something. And duplication is not always apparent. When you take single-responsibility as far as you can go, you not only isolate all of your moving parts but you maximize the opportunities for reuse.
And it goes even further than that. Your large functions likely have a few dependencies, at least. Those dependencies will make your functions more difficult to write tests for. And your test cases will be more complicated if you have larger functions because they are doing more than one thing that needs to be captured.
Readability is more about expressing the intent of code. You can do that in "long functions." You can express the intent of code while making it extremely compilcated. Hell, just add lots of verbose comments and your code will be more "readable."
Since you seem to be one of those people who has a stick up you about design patterns (your last comment about AstractFactorySingletonViewModel ... who hurt you?) I will offer you this piece of food for thought: the purpose of "best practices" and design patterns is to SIMPLIFY code. If you ever see a misapplication of them in the wild* then what you are witnessing is not "over" engineering ... it is POOR engineering. Consider that before throwing the baby out with the bathwater.
* I rarely do, so I often wonder to myself if this is a made up problem by lazy devs who don't want to actually study theory. But I do hear that this occurs from time to so I'll take you at your word that there are people out there that don't know how and when to actually apply design patterns properly. The problem is the misapplication, not the patterns themselves - which are just common solutions to recurring problems. Do you not think DRY is a good idea?
My friend and I were working together, when one of our seniors came over and asked us for help figuring out how to modify an algorithm that someone new on the project had rewritten. The algorithm was 20-30 well-commented lines of code, until the rewriter decided that it lacked flexibility. It had been rewritten to use a Singleton Factory to instantiate an implementation of an interface that itself delegated every snippet that could possibly be conceived of as a function to one of several different classes (Policy, Actor, Dispatcher, etc.). Every function was three lines or less, and all were entirely illegible taken in part or in whole.
I’m glad that team member got to show us how many times he read about design patterns and how much he knew about OOP, but it took four engineers (we later roped in another) to venture 15 layers down the call stack of virtual functions to figure out where a simple constant came from. Quality code indeed.
What you gain by making every function responsible for exactly one thing you lose through more abstraction and worse local context.
It's always a balance.
That's a non-sequitur. The word "extreme" describes matter of degree. If something is "good" then it does not follow, logically, that "extremely good" becomes "bad" just because it was "taken to the extreme." As you yourself said, context matters.
Then to say "it's always a balance", a balance between what? Good and bad? Simple and complex? What are you talking about?
We are speaking in the abstract, necessarily, when we speak of "small, single purpose functions" because we are taking the context out of it. Can we agree that the simplest solution should always win? If so, why?
After 25 years of developing software professionally, I have seldom seen someone apply design patterns, so-called "best practices" and engineering principles to a point where they shot themselves in the foot. It happens, we are all human after all, but in my experience it is not a huge problem.
What is a huge problem, is that time after time we inherit projects written by very decent hardworking developers who can get something that works out the door quickly but don't know how to write it in such a way that it is easy to change over time. We see massive files, massive functions, lots of cyclomatic complexity, terse variable names, duplication, tight coupling, near zero modularity and very poor separation of concerns. This is the norm, not the exception.
And yet lately, I have seen a trend that is a knee-jerk reaction against decades of work done to address this problem and it should be concerning. If 75% of the projects I worked on were tech-debt free, and we could identify actual examples of software where people wrote it so abstractly that it was incomprehensible then I would be on the same side. But what I see in the wild is not examples of people going "patterns happy" or taking things to a point where it starts to become complex rather than simple. What I see are developers who have never even heard of design patterns, or they learned one and then misapplied it everywhere because suddenly every tool looks like a nail to them.
Therefore my position is that we should be preaching single-responsibility and small functions more, not less.
> Then to say "it's always a balance", a balance between what? Good and bad? Simple and complex? What are you talking about?
A balance of the extent that you follow the rule/principle/pattern. There is a reason we have both WET and DRY.
> Therefore my position is that we should be preaching single-responsibility and small functions more, not less
Fine, but you're preaching to the wrong people here.
Even DRY, I've seen DRY cause test suites become hard to work with. Rules of thumb sometimes get pumped really hard and I think it causes people to over-use them. The rule becomes the goal rather than something more material like UX or DX. This code is DRY, DRY is good, therefore this code is good - I hear arguments like that quite often. But your take feels very grounded.
It doesn't really work if I'm breaking up a function body into a bunch of smaller functions that will never be called from anywhere except the refactored function. I've just moved the complexity from a large function body into a bunch of smaller functions and the calls to them - but it's still the exact same logic, with the same complexity (but perhaps harder to read, due to all the jumping back and forth).
It was 3 layers
- API interface (unpacking objects, etc)
- Business Layer
- DB/System Interface.
The business logic changes could often be totally contained, without updating the API or the database. It would be so obvious what layer needed to be edited, and the problems could be fixed so much faster. Rarely did a commit change two layers at once.
You can easily handle AuthN in the API layer, AuthZ in the business layer, and then access to a db in the system layer. Assume API is un-authZ’ed, assume the system layer is fully AuthZ’ed, etc.
There's a tradeoff between understanding the function itself and understanding how/where the function is used (and therefore understanding what will break when that function is changed).
The best heuristic I've got at the moment is roughly the square root of the size of the module. So, for the 10,000 line module, my vague instinct is that I'll have one hundred functions approximately one hundred lines each. That's not a hard and fast rule, just an observation about tradeoffs and I would definitely expect (quite possibly even most) functions to differ significantly without losing sleep over it.
Splitting functions is tempting because it makes each individual function seem simper, but the higher cost in complexity and maintainability is not as directly visible.
Shorter functions are fine when appropriate, but length by itself is not an indicator of quality.
What does someone need to have in their mind to understand this program?
Then time happens and then the code is adapted and refactored and more features are added, then the original gem of mental model is hidden by hundreds of files and the algorithm is split into 10s of files for the little bit of wisdom that makes the whole program understandable to be occluded.
This is why I like full code listing examples in documentation so much. A small self contained example - with import statements and instructions on how to build it if necessary and dependencies mentioned, I can get started extending the program to do something that I want.
Strangely, I find leetcode solutions or algorithms easier to understand than big enterprise systems or codebases where people do things the right way, because the core algorithm is right there and you cannot miss any detail.
I've enjoyed using OpenGrok a code indexer and viewer to turn any codebase into a wiki.
When I come to a new codebase, I have no clue how people have split up the code into many files.
The DWM codebase is fairly straightforward to read because it's all in one file.
I want to read about your code rather than the code itself if I can help it, I prefer reading documentation or deep dives or tutorials into code than a large github repository with hundreds of files.
I really liked the trend for a while in web frameworks where the code was side by side the documentation.
And some positions seem quite strange. Not all function names will relate to "the problem the user wants us to solve for them." What does the user know or care about a function like atoi() or malloc()? And is it really unnecessary to ever write comments with readable code?
No, i don't talk about the "haskell style FP", i'm talking about FP thinking.
It could help you clean your mess in a productive way, and i call it the true craftsmanship.
This distinction is perhaps something we should advertise more.
I don't know if things have changed; but a decade ago when I was in uni we had some hardcore academic professors that loved functional programming and tried to teach it to us through languages like haskell. Which was wonderful stuff and interesting, but worlds apart from some of the practical stuff that most devs do.
I used to see big names in software advocate for more functional programming and I didn't get "it" because I thought there were talking about haskell, F#, and advanced type theory. It took me a long time to understand they were talking about functions as first class objects where you can pass in a anonymous function or lambda, or whatever your language calls it instead of all this boilerplate for a separate private class like you would in old school java. Or thinking about things in terms of data and pipelines, for parallelization. And balancing this all out to get an intuition for when to use what without being restricted to any one style.
You need to test but if you're testing on the scale this guy is it's a sign of bad practice.
However, this is a fantasy.
>I’m obsessed with making my code readable, even to non-programmers
This is a plainly impossible goal, because programmers themselves don't agree what "readable code" is. The problem is that sufficiently useful code always exists in a context where tradeoffs are inevitable.
When the universe aligns to help us forget this, the success can be intoxicating. Wouldn't it be nice to live in a world where all projects can be like that? Software Craftsmen then get seduced into Software Masturbators.
I don't think the article does a super job of explaining itself or making visible this conflict, but I think this is one of very few pieces out there that talks to one of the realest most core splits in the field, that causes some of the most tension & agony between different people & different parts of orgs trying to work together.
This those of us close to the machine, it is a conflict that sometimes feels like it has few wise, knowing, pro-Craft allies in.
Too frequently this attitude is just a bad cover for acting like your shit don’t stink.
And sure, the two probably cross-paths a lot. Your "counter" seems to imply that my stated problem should be ignored & isn't real, that your problems trumps my problem. I feel we should take a broader look that can accept both problems as perhaps real.
Who suffers from each problem? Who has allies as someone suffering under each condition? What are the risks of these problems? What are the pushbacks? Code-tyrants suck, but I think the org generally disfavors them, that they become unpopular on their own accord. Where-as people feeling the org is being short & persecuting craftsmen, they have other devs to gripe with, but they also alienate other devs wanting to just move fast, wanting to prioritize industrial output, and managers who see them as obstructing. In both cases we have devs who get seen as obstruction, and the line of whether there is over-concern tyranny or not shapes the perceived moral arch of their concern. Personally, I think the code-tyrannts often get just-deserts, whereas people with real intents end up victims.
Living with code tyrants suck, but I tend to think it's somewhat self-correcting, where-as orgs never have to learn, never have anything actually incentivizing them towards awareness.
I'd also say, rarely is there much attempt to actually negotiate with or address code-tyranny; people tend to just suffer, and the hard conversations/honest candor about the code-tyranny often never occur. Where-as developers often wear their heart on their sleeves about their commitment to quality & wanting to do a good job, and their please/cries are often ignored, as the org carries on from it's far-off unseeing place.
Testing adds some cost, so there's a tradeoff. You want a decent ROI on your testing efforts, otherwise you're wasting time and money. Opportunity should not be ignored.
Software doesn't have to be zero defect, it has to be low enough to be useful.
Have a hundred-line function, when it makes sense. Try it out; it won't kill you, or your team, as long as it is well-documented.
....
the solution space of software useful to humans is barely explored. imaginative, unconstrained, passionate builders are for great good.
collaborative software implementation for salary in the typical cost center environment is always going to be a tragedy of the commons. this is actually fine.
There are many obvious examples where it makes excellent sense to refactor a complex function into several smaller functions. I get frustrated with the spirit of this post because I honestly have never seen anyone challenge this.
However, I frequently see PRs which appear to be breaking a longer but repetitive function into a large number of mostly single line functions, apparently with the single goal of achieving checklist compliance with the folk wisdom. In these cases, we've devolved from a list of (for example) simple name mappings with the occasional downcase() (for the bold!) to having to jump back and forth between multiple points in the file, which has now grown by 80-150% to account for all of the begin/end sigils (language/lint dependent).
If you're converting a JSON string to a dictionary and then mapping a large number of attributes, moving from
object.foo = json["foo"]
to
object.foo = extractFoo(json)
with a matching three line getter you have to scroll 2-3 pages down to read and does the exact same thing is not a win.
TL;DR context matters and general rules only help when you don't treat them like hammers.