Programming is boring art
ahsmart.com
ahsmart.com
But I would like to address a tangent: extracting configuration from code. Let's take the article's example and change it a little. I'll decompose it, since it already has become a function, but often the first step is just a series of statements. (I will also port it to javascriptish pseudocode, since I don't know Julia.) So this might be your first draft:
response = HTTP.get('https://www.example.com/');
if (response.status !== 200) {
email('someone@example.com', 'something is wrong');
}
url = 'https://www.example.com/';
ok = 200;
to = 'someone@example.com';
msg = 'something is wrong';
response = HTTP.get(url);
if (response.status !== ok) {
email(to, msg);
}
Is there a name for this kind of refactoring?
In general, it is best to create small functions that takes few dependent variables for the intended flexibility. The first judgment here is to decide whether a value should be a constant or variable. Constants would then be declared at the top, and variables come in as function arguments.
You reminded me that another pattern to handle this is to put some of these values as configuration parameters. For example, the email alert may need to be sent to different people for different environments - DEV, QA, and PROD.
You can extract config to various degrees. My example is perhaps the mildest: just group it together, near the beginning. You don't need complex frameworks. In fact this technique first caught my attention when I was reading other people's shell scripts.
The next step would be to move the config to its own file. This is very common in Linux. You have some tight binary, that is compiled and hard to change. But then you have config files, often with dozens of options. It's a nice way to do things.
The next step would be to move the config to a different level, no longer in a simple file. This is often either a relational database or environmental variables. Linux commands use environmental variables along with their config files.
For very large systems, this technique is the same in kind, just different in degree. You get fancy specialized systems, like Hashicorp Vault, to store your configuration, at least your secret ones, like passwords and stuff.
I'd call it introducing a few variables.
It does one well-defined thing, and does it well! That's the good kind of 'inflexible'.
> A skillful programmer looks at the problem and says, hey, why don't we build an abstraction layer to make it more flexible?
In this example, the additional complexity just doesn't seem justified. You ain't gonna need it. [0] Unnecessary abstractions cause bugs, bloat, and baggage. The article then goes on to acknowledge this, but I really question the chosen example.
> Just having these few abstractions opens up a lot of possibilities. Why? Let’s suppose that you have 3 different kinds of fetchers and 3 kinds of notifiers. Now, you can easily compose 3 x 3 = 9 different kinds of health checks.
You've just introduced an exponential explosion into your testing burden. The pay-off: additional functionality that might be useful later.
edit On reflection I suppose it's not a 'exponential explosion', it's merely a 'multiplicative explosion'.
> Enjoy your programming life. It's a fun art!
The article doesn't go into detail on what it means here, but the more serious the software work, the less the artistic mindset applies. [1]
I think the most sensible approach to software engineering is to draw satisfaction from solving a challenging problem well. Things like 'programmer freedom' strike me as false idols.
That said, I'm a sucker for elegance (a famously subjective commodity), so maybe I'm a hypocrite.
[0] https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it
[1] https://www.fastcompany.com/28121/they-write-right-stuff
What happens when an "abstractnotifier" blows up? Where does the error happen? what are the possible side effects? What systems might be impacted?
What happens when 'alert's start failing? Is it queued/buffered? Do I need to worry about OOM errors on systems? Will it retry?
I lead a team of developers - This kind of abstraction tends to slow everything down. The only place it fits is during a refactor of software delivering real business value already. Basically - You can do this if you know exactly what your requirements are, and this nicely solves them.
If you do this before you understand EXACTLY what you need, you're going to be wrong. You will abstract the wrong pieces, with the wrong assumptions. I promise you. Then later you have to rip it all apart anyways, except now instead of 4 simple lines with clear consequences, you have a bloated monster of bad assumptions.
The main problem with this is culture imo. I agree 100%, but it is so hard to fight against the idea that "real engineering" or being a "good developer" means tiers of abstraction.
To me, being a good developer actually means understanding what your operating parameters are, and then writing the smallest amount of code that satisfies the business requirement. Sometimes (a lot of times!) this DOES mean you have to go back and refactor or completely throw away code. By _design_ you chose to only solve the immediate problem. However, culturally, this is seen as a failure rather than a success.
This statement seems rather inaccurate and misleading. What you thought about 'multiplicative explosion' is the number of integration tests required. Generally, software testing focus more on unit tests, and so in this case, the number of tests is still linear.
> The article doesn't go into detail on what it means here, but the more serious the software work, the less the artistic mindset applies. [1]
This claim seems to be too opinionated. I doubt seriousness makes a difference. I have seen artistic/creative design in some very serious projects before.
I don't follow. The new function has 9 different modes of operation to be tested. I don't see the relevance of the distinction between unit tests and integration tests.
The alternative is to decline to test them all, of course, risking a trade-off between bugs and baggage.
Even in this trivial example, there's a bug resulting from inadequate testing. [0] Perhaps in this example we might be confident enough in the independence of the two parameters that we wouldn't bother testing all 9 combinations, but my general point stands.
> I have seen artistic/creative design in some very serious projects before.
What kinds of projects? Avionics? Formally verified systems?
Formal development solutions like SPARK, and informal frameworks like MISRA C and other restrictive coding standards like [1], greatly restrict the programmer's freedom.
The alternative is to decline to test them all, of course, risking a trade-off between bugs and baggage.
Even in this trivial example, there's a bug resulting from inadequate testing. [0] Perhaps in this example we might be confident enough in the independence of the two parameters that we wouldn't bother testing all 9 combinations, but my general point stands.
I think what he means is that, in unit testing, you only test code units in isolation to each other, with the tests assuming the other parts that your unit-under-test interacts with behave exactly as the unit-under-test expect: If you had n producers that could plug into one of m consumers, you'd test each producer in isolation against a mocked consumer and each consumer in isolation against a mocked producer. That will give you n + m unit tests, which is indeed linear.
Of course this hides the fact that in practice, you'll have n * m different modes of operation, so you'd need n * m different integration tests if you actually wanted to test each possible interaction.
You can get away with only testing units in isolation if the interface/API/protocol between producers and consumers (in this example) is well-defined and rigid enough that implementation details in one part really have no effects on the other part. I think there is lively debate how well this is possible in practice - in effect, this is the whole point of standardized interfaces. However, even attempting this is often a lot harder than it appears and therefore shouldn't be done if not necessary.
That being said, I think it's important to keep in mind that you can almost never write tests for all possible configurations that a program could run in, if your program is non-trivial. You get combinatorial explosion pretty fast, even if you don't use any abstraction at all. And after all, the whole point of a program is that you can feed it with novel inputs without knowing what the exact output will be - becasue finding out that output is what you wrote the program for. So you'll always have to think about classes of inputs and restrict your tests to (hopefully representative) examples of each class.
Art is about giving other people rich, engaging experiences that distill and/or highlight some element of relatable human emotional, psychological, and/or sensual experience.
The more unusual, original, and insightful the experience, the more interesting the art.
Doing something difficult with elegance is not art. Solving problems is not art. Complex but comprehensible systems that have "beauty" are not art.
If there's no emotional or sensual content, there's no art - just technique.
There's nothing wrong with technique, and art isn't better or worse than technique on its own terms. But they're different languages with different goals.
Calling software abstraction "art" is like watching someone who has discovered a for loop trying to persuade you they're an expert developer.
(In very rare circumstances I might consider a program to be art, like the original metacircular Lisp evaluator, even if the author’s express intent wasn’t to appeal to somebody’s emotions.)
I think art is contextual. Some art comes with that context built into the mind of the observer, like paintings or music, or something that tells a story - and there are many, many ways to tell a story.
Other art has its context declared for the observer, like Duchamp's 'ready-made' sculptures.
This is a great definition (and this is coming from someone who has studied it for decades).
I wonder, though, if you're being too hard on the writer. Art originally meant the same as skill or craft (https://www.etymonline.com/word/art#etymonline_v_17037). You can see it also in the word artisan, which would likely be applied to a cobbler or carpenter. I believe you are defining the fine arts, which more specifically refer to art purely for the sake of emotional expression, as opposed to the more practical arts, like cabinetry.
Nowadays the sense of the word art has narrowed more to what you're saying, to what our forebears meant by fine arts. But it is not always used that way. Many times I have heard people refer to common, practical activities as requiring some "art". Or they might say that someone who does it with exquisite taste has elevated the activity to art (like, cooking). This is even though it wasn't about, as you say, distilling experience into some other medium.
People loosely use the word art any time that an activity presents some options, any time there is a choice to be made, and a much better result happens when you choose well. By this definition, almost anything could be called an art, or at least could involve art. The only exceptions would be things that are 100% formulaic, like maybe working an assembly line (even then, I wonder if assembly-line workers distinguish some room for art in their duties).
The two meanings continue to intermingle, because all of the arts take craft, and all of the crafts take art. If you want to be a composer, painter, or writer, there are skills to master (the musical scales, paintbrushes, sentence structure). If you want to be a carpenter, chef, or architect, there are always choices to be made, every project is unique in its own little ways.
My point is, your definition is very useful to anyone attempting fine art. But I wouldn't attack people who use the word more broadly (and this is coming from someone who likes writing and who often is irritated by imprecise diction).
In design you work towards a desired outcome, usually something that adresses a group of people, and therefore a set of characteristics. The outcome can be repeated, or changed in a predictable manner. That what lies between the outcome and your intent is what design is obsessed with, process, templating, decoupling from environmental factors, including your own personal characteristics. Sounds like functional and good OO programming right?
Art is the opposite, it's obsessed about capturing unique outcomes, originality. This means that what lies between the outcome and your intent is also about the opposite, coupling unique personal and evironmental inputs.
Performing arts, or arts and crafts are more like design, outcome driven, it's only that they can't remove human inputs because those are also expected to be there.
Some of the loveliest art wasn't created with any care or interest in anyone else ever seeing it-- but just because its creator loved it for what it was and felt it needed to exist.
If you're a programmer and have never seen true art in a program or its underlying algorithm-- some ingenious machine which moved you with its elegance or serendipity-- then I feel a little sad for your sake. It's not necessarily common, but its out there if you open yourself up to it.
Likewise many people currently use "Corona" instead of "CoViD-19". We still communicate just fine. Still, precise use of words is necessary in technical documentation and a blog post might be considered as such. Maybe not though.
art /aart/ n. 1 a human creative skill or its application. b work exhibiting this.
There are assemblers, people that can follow a set of IKEA instructions and end up with a reasonably stable bookshelf. The average starting developer out of college is at this level.
There are journeymen programmers, they have a few years experience, understand some of the nuances, know a bit about what doesn't work, can use the word "strategy" both correctly and ironically.
Then there are master craftsmen, they understand the material they are working with. Know when to use an IKEA wooden dowel and when a custom joint is required. They build cabinetry that can be on display for its own inherent structure and leverage the natural capabilities of the material.
The same applies to programming, both in the small and in the large.
I agree that most programming is craftsmanship.
Code like the Obfuscated C Contest is art.
The code that orchestrates your cloud servers is more like engineering.
Prototypes in language research is science.
https://github.com/solvespace/solvespace/blob/master/src/srf...
If you have a basic understanding of spline and NURBS surfaces but never knew how you might actually implement that, well there it is. Incredibly readable IMHO.
https://www.youtube.com/watch?v=tX4H_ctggYo
Discussed here:
Beautiful code to me involves the creative composition of parts to form a whole in a way that seems obvious only in hindsight, but takes a form of "genius" to come up with.
I've written a few compilers and seen how the way you model the problem radically changes how easy/hard the implementation of "features" (as opposed to said "model") becomes. That, to me, is programming beauty, therefore, art. Code that model the problem so well that it brings up clear as water problems you hadn't event thought existed. Code that makes apparently difficult transformations trivial. I wish I could give you some examples, but I can't think of any right now. If you know of some, I'd love to look at it.
I agree that choosing the right definitions and structures can make more problems solvable. That's a major driver of data structure choices - the things we build need to support the things we want to do, and getting that right is also part of the art.
No he doesn't
Small critique: s/he
I think there is something to say for not overdoing it, but for not mindlessly implementing only the thing either. Thinking about what is the next abstraction that includes the functionality your want to implement and also is useful to implement anyway, because you already expect the need for flexibility in a certain direction.
It depends probably on what type of software you write. Just a few procedures in a legacy code or a little command line tool, standing on its own for example.
Certainly implementing only "the thing" gets the job done though. Perhaps doing more is doing more than what one is paid for.
You started with a simple solution that nicely handles the current needs.
You then over-engineered a solution so badly that you haven't even noticed the 'url' param isn't used in the last method you've shown.
You went from simple enough a junior dev can make meaningful changes, to bloated enough the original author is fucking it up.
Don't do this. Wait until you actually have a REAL use case for adding complexity.
I think he wrote 'site' instead of 'url', so I'd call it a typo, but... I agree with what you wrote. For me, the article transports a wrong message. It's a thin line between YAGNI, KISS and OMGINTWNT[0]
[0]:"OMG, I NEVER THOUGHT WE'D NEED THAT!" - made this one up, sorry.
The intention to illustrate the power of abstraction. Based upon yours and other people's comments below, perhaps it wasn't the best example.
I did correct the typo about 'url'. Best regards.
Loggers are examples of incidental complexity. There is nothing fundamental about sending the warning by email vs slack vs irc. The way to deal with this is not to keep adding more and more options, because you end up with combinatorial growth in the number of interactions, it's to make it company policy that you only use email and discipline anyone who doesn't.
I agree with this but I would put it somewhat differently. To me, notifications are a separate service, and whatever options there are for notification should be orthogonal to whatever is doing the notifying. If company A's policy is that notifications only go by email, nothing else, while company B has email, slack, and irc, that's for their respective notification services to worry about, not for each individual application that is using the notification service. To the application, it should be send_notification(content, recipient) and that's it.
And in fact, this is what Arthur Whitney (creator of kdb+/q and Shakti) does. Every new version of k is started entirely from scratch. No code reuse, no libraries, no sharing.
I believe in reusability with software engineering. The reason why legos is so much fun is because you can compose and build things out of simple shapes.
I also agree with you and others in terms of simplicity. I guess what I failed to communicate in the original post was about the judgment that is required to make the right decision, and that caused quite a bit of opposing arguments.
I believe that the right level of abstraction is required for any non-trivial system. Life is full of balancing act - like yin and yang. Too little abstraction is also costly and leads to code bloating and maintenance nightmare.
My real problem is that abstractions are Expensive with a capital E. That typo happened because you had to think harder, about more lines of code, for a longer time. You're spread thinner, and the amount of time you spend actually reading each bit goes down. It's easier for errors to slip in. It takes more time to reason about. And worst of all, most people are genuinely bad at coming up with good abstractions - It turns out it's really hard too.
Now - Once a problem has reached sufficient complexity - you need them. But they really should be avoided until you're AT that point.
Using an abstraction too early is overfitting to your current problem space.
To make a more genuine criticism than just pointing out the typo - What happens if it turns out that your next health check 'fetcher' needs to emit events, or requires a callback, or doesn't use a url but a redis connection string?
You haven't avoided the need to change "check()" in any of those cases.
The abstraction just leaked, and now you have to understand every last bit of all that code again. Which is another chance for small errors to slip in, which they will do, because programmers are human and we fuck up ALL the time.
From my experience, the best way to avoid fucking up is to just not do anything you don't absolutely have to. Avoid adding lines, avoiding adding more things to understand, avoid adding complexity, avoid adding abstraction.
Do the job you need to do, and JUST that job. But hey, it turns out that's Boring!
I wonder if programming fonts could achieve that - like the original author could specify a font for a portion of the code eg. a “destroy_context” or “drop_connection” function could have a scary look.
https://github.com/tk3369/BoxCoxTrans.jl/blob/master/src/Box...
This certainly also depends on the language you are using and the ability to define arbitrary in/postfix operations.
It’s also rather annoying to type them, but that’s also probably because I just haven’t developed a workflow for it.
That's true. I remember hitting C-c and C-v a lot... XD
I wonder how mathematicians do this, I suppose they use latex notation most of the time, inputting ASCII which is formatted as the symbol instead of looking up tables
The consensus was that I'm a writer.
Also there's one thing I ask everyone seeking to get into programming: "are you resistant to boredom?"
This appears to be a good predictor of future success.
I was thinking of the former.
Glad that I bore easily and programming is not boring.
Maybe relevant talk from DHH coming to the same conclusion: https://youtu.be/9LfmrkyP81M
A skillful programmer doesn't add layers of abstraction unless there's a good reason or need to.
Yes, pedantically this approach is only used in certain circumstances. Then, a skillful programmer has a ready set of skills. And so on.
Same applies to any text written. Yet, few people would argue there aren’t any artists among novelists, screenwriters, and poets.
These are all skills I'm pretty sure not mentioned in most programming books/blogs, much less schools.
Boredom comes from within. Pointing at a thing and saying "that's boring" is a conflation of what's really going on: "I haven't learned to enjoy myself in this context yet and am blaming the context for it."
1. Paradigm purity is worthless 2. You ain't gonna need it. 3. Less complexity is better.
You just broke all of them.
Abstraction is a sort of entropy / Shannon / Kormogorov issue. So let's discuss it there not semi-liberal-arts school fogginess.
Folks let's please keep science in science. One day if QC gets commercial all these flowery articles will disappear. QC is math based. We'll need to deal or get left behind, and by then all this handwaving won't matter in the least.
Now, the other bad of software dev is that today's work is generously semi-formal at best in commercial settings. Not because CS is art, but because requirements gathering, human factors, corporate culture are strongly reactive in the software realm, where mother nature is directly absent.
Here we could help each other to understand management and management us to move this whole ediface along. And here less technical articles ala HBR work well.
Here's a contrived example:
if (a < b && c == d)
createUserProfile();
if (userIsAuthenticated())
createUserProfile();As the OP mentioned, beauty is created by abstraction; an ability to look at the bigger picture and make sense of it. So I agree that abstraction is the most interesting thing in writing software.
That's why we like entertainment - because it cuts out the boring bits :)
The art part is more interesting. Is there art to someone's life? I'd say yes, but that's because I'm artistic.
If you ask most people, they'll probably be confused by the question. Is your life 'boring art'? 'Huh' is the answer I most expect :)
>I can sympathize with a lot of nihilist outlooks on existence, but just because a lot of these might be true, they're not the only ways you can view your flesh or your place in these patterns, as an animal or as a cog in a machine. There are other elements to existence, like imagination, that is a good example. Emotions are a great vehicle for building your own universe and reality, where your function as an animal or cog in a machine doesn't force you into some rote role that drags you into this stripped down reality of raison d'être being purely utilitarian and unchangeable. You mentioned imagination, but emotions make things real. Does it really matter if your imagined reality isn't concrete or factual? What matters is your happiness, right?
That adrenaline action movie junkies think moment by moment life is boring, doesn't mean it is.
I don't think Hardy's take really stands up today. There is at least one mathematical fact which has been proven through exhaustive computerised tests, and which no-one has (yet) been able to prove without using computers.
Ugly? I suppose so. Is there a 'place for it'? I don't see why not.
https://en.wikipedia.org/wiki/Four_color_theorem#Proof_by_co...
Perhaps a nitpick, but 'correct' deserves a mention somewhere.
As others have said, elegance in code should generally align with efficiency and correctness.