Coder's High – The Intense Feeling of Absorption
slate.com
slate.com
The productivity gains seen from things such as Kanban, TDD, and CI/CD can be easily explained by the science of flow. I've given several conference talks on this and they've been well received (asked for encore presentations).
As a quick teaser, think about how CI or fast tests contribute to flow when in order to achieve flow these three criteria must be met:
1. The goal must be clear 2. Feedback must be immediate 3. There must be a feeling that our capability is balanced to the challenge at hand
I do think that the author doesn't understand Flow fully. The ideal state of Flow is when there is a concrete challenge with immediate feedback, which coding is great at providing. The other activities he compares coding to don't have such strong feedback or concrete challenge.
The only activity that compares with coding for me is rock climbing. While it is a physical activity, it has the same kind of concrete challenge and immediate feedback. It is totally immersive.
It's the combination of feedback, decision making and then successful completion of the challenge that really combines to create the feeling.
Thinking of the problem, planning your response, and then executing in quick succession repeatedly.
Especially if it's off-piste but an area that you're familiar with. You plan out your run multiple steps before you start, and once you're on the hill you have to be flexible enough to change course and replan on the fly for another multiple steps ahead, all while you're barreling down a hill.
And then there's the moments of sheer terror when you see snow sliding past you, or when you come up on a recent avalanche 1) have to avoid the packed ice and 2) look out for what caused the avalanche.
You're absolutely right - I get a kind of rush from coding that's similar to skiing off-piste. Not quite the same but not as different as one would think.
A few tasks, though, have been serious blockers. When I don't understand my tools, and they fail in undocumented ways, and the relationships between presentation and business logic aren't clear, it actually hurts my feelings. It's as if the machine itself is judging me and reminding me that I'm not good enough.
I'm insanely jealous of people who can do stuff like this. I hear stories of people like John Carmack locking themselves away in a room and finishing an entire project or implementing an entire feature in a night or over a weekend. In contrast it takes me days to finish anything, I just don't have the ability to concentrate or care about something that intensely.
Now I understand some stories are embellishments and someone like Carmack just thinks on a different level than most people, but is there a way to teach yourself to be able to engage in these marathon creative sessions or is it something you must be born with?
Anecdotally, I think the answer is yes. Concentration is a skill you can develop like anything else. The kind of high with programming for me is similar to that experienced when rehearsing/performing music.
I used to perform on a competitive drumline and we would have 8-10 hour rehearsals on the weekends (usually in 3 hour blocks). One of the main ideas expressed was that if you want to perform at a very high level, and do it consistently, you must always rehearse at performance levels and be hyper attentive to details. These rehearsals were mentally exhausting. But over a few months, my ability to concentrate and to quickly jump into a flow state improved dramatically.
One thing that surprised me - after rehearsing with a metronome for that many hours a day and usually at the same speed, say 120 BPM (beats per minute). When we would change speeds, faster or slower, even a 2 BPM different (118 BPM or 122 BPM) felt noticeably different than 120 BPM.
After I stopped performing that group, my ability to distinguish these small difference in tempo regressed. I can't do it anymore. A wide gap like 110 and 120 is noticeable (probably even to non-musicians), but not only a 2 beats difference.
If you want to try this yourself, get someone around you to go to http://www.webmetronome.com/ and set it to 120 BPM and 122 BPM and see if you can correctly guess which one they set it to.
I think Carmack (and people like him) grew up in the proverbial basement, where they could spend hours and hours on end with their hobbies without any social obligations; this is where they developed that concentration, and that state of mind and environment is where they return to if they need to work on a big or complicated task.
> is it something you must be born with?
A motivation to reach this state, to keep trying and pursuing it, may be a fact of your nature. Or, it is an understanding of what your work will create. Fame, money, fulfillment – extrinsic motivators that drive you towards a definitive conclusion.
To meditate in your work might just require you to find your motivator.
For me flow is achieved most effortlessly when the problem is interesting and I have the tools necessary to attack it.
Extrinsic motivation is simply a non - factor in this case. To the degree that it may or may not exist and is immaterial to the labor at hand.
On the purpose of a tight feedback loop. The magic of the repl is a perfect implementation of a flow like feedback loop. Unfortunately my language of choice doesn't utilize it.
Hiding things also helps, things in the visual field that you have to work on or are interesting are literal distractions.
> The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means. Whenever you parry, hit, spring, strike or touch the enemy's cutting sword, you must cut the enemy in the same movement. It is essential to attain this. If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him. You must thoroughly research this. [1]
I've been able to experience the flow for as far as I can remember. It was years before anyone even told me it even had a name. I've read the wikipedia article [2], but not the actual book. So I'm not an expert. But as far as I can tell, reaching the flow involves knowing your goal, closing the feedback loop, and pouring all your attention toward the goal.
For example, consider reading a novel (written in English). Someone who can't read English obviously won't be able to read it. But for someone who speaks English as a 2nd language, sure, they'll be able to read it. But more slowly since they have to think about things which are non-central to reading. Things like syntax, spelling, idiomatic expressions, irregular verbs, conjugations, etc. I.e. the implementation details; the cruft. The non-central things get in the way of the important things, which include semantics, plot, character development, irony, point of view, theme, tone, mood, etc.
Personally, I see novels (and sometimes non-fiction) as a gateway. As in a gateway into another universe. Because for one reason or another, I don't struggle at all with piercing through the syntax or get the slightest bit fatigued (though technical documents are different, those take effort for me). But I think that for some people (even for whom English is their primary language), text represents a barrier. "Why not just watch the movie?" Rather than using the semantics to simulate this alternate universe, they get bottle-necked into trying to translate the glyphs. So where I see as an epic space battle, others see an intimidating bulwark of text.
Because some people have this mental image, guys like Chris Granger have been trying to figure out how to translate a wall of text like a C source file into a novel's equivalent of a movie. We've gotten a lot of cruft out of the way with higher level languages, but it's not enough because programming still text based. I don't know if it'll work out, but that's the ideal.
The second thing is the feedback loop. Feedback has to be immediate. The feedback also has to be exact. I.e. you've got to know your craft better than the back of your hand. A musician will know in real time how well they are performing. They don't just notice the obvious mistakes, but even the minutia like jliechti1 was discussing. And they have to be confident about knowing that they indeed made a mistake so they can immediately act to correct it. The reason feedback comes second is because if you're still bushwhacking through a jungle of cruft, the feedback loop won't ever feel instantaneous.
Another analogy is being able to unwind the recursion without having to plug and chug each step. There's simply no effort expended. You can model the function well enough that you can visualize the path it takes, like the chess master could in the article. You simply cut through to the base case.
I like the Musashi quote because it perfectly embodies the first two items. The goal is to win the fight. But some people get caught up in thinking too much about the techniques. E.g. parrying is useful. But if you don't understand how to take advantage of it, it's mental cruft [3]. Additionally, if you're not confident, you'll hesitate and probably die. The stakes aren't as high in programming, but the same idea applies: perfect mastery is only a prerequisite to achieving the flow.
Third is the focusing of attention. It involves a state of mind where you're naturally able to block out all distractions, and focus everything on the task at hand. This part is a little like falling in love: you can't force it, you have to set the right conditions and pray things simply work out. This part is really frustrating compared to the first two items. You can always review your specs to better understand the goal and constraints, and maybe philosophize a bit. You can always practice and acquire skill in order to better fine-tune the feedback loop. But when it comes to focusing attention, all you can do is remove distractions. Which sounds ironic since "why would I want to block out distractions if I've removed them already?" I don't know how to explain it better, but that's how it works for me.
Interruptions are the worst. If you go out to lunch for an hour but you're still pondering the problem, then flow won't necessarily break. But if someone asks you to help them move furniture for 3 minutes, that forces a context switch and everything collapses like a house of cards.
Cthulu below mentions something about growing up in the proverbial basement. I think there's a lot of truth to that. Basements = zero distractions. Maybe childhood better conditions some people for the flow than others. But I did notice that once I heard the name "flow" and realized I already experience the emotion on a common basis, I was able to (try to) actively graft it into other activities. I think being able to actively seek it out helps. So I think it's at least somewhat trainable.
So here's a recap. In order to achieve the flow: master your craft; tighten the loop; avoid distractions.
[1] http://www.uvm.edu/~asnider/IDAS_2011_CD/Teachers/Steve%20Ll...
(Also, am I the only one who gets slightly annoyed with the word "coding"? It seems to ignore the fact that most problems we sit around solving revolve around systems, integration, framework intricacies, logistical/timing issues, and stuff like that. The code is just the instruction set. Then again, maybe I just want my job to sound more fancy and shit.)
headphones just don't seem to be a strong enough do-not-disturb indicator
One thing I've noticed is that even if people don't actually interrupt me, the expectation that they might at any minute prevents me from fully concentrating on the task. Having some sign to indicate that for the next, say, 30m I should not be interrupted, and having people respect that, would probably help significantly.
On the other hand, the fact that people are constantly needing me is partially my fault, since there's stuff I could empower them to do by themselves with a little automation.
I couldn't do an all-nighter of that awesome zen-matrix-coding-until-the-birds-start-tweeting without it, but for noisy work situations it also helps to get into that mode.
Lock in the headphones, put on some tunes and 'focus mode' and someone can stand next to you for 5 minutes until they realize you zoned out and they need to tap your shoulder to get your attention.
I haven't found the perfect key to good coding music, but these are my 2 favorites:
* http://soundcloud.com/crussen/crussen-monday-funk-session (a 4.5 hour mix)
* The Fugees - The Score (full album)
I recommend Anjunabeats and Above & Beyond if you are looking for melodic/uplifting electronic music that avoids potentially distracting lyrics and jarring sounds.
"The delight of working in such a tractable medium. The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of imagination."
I love that quote! More here: http://henrikwarne.com/2012/06/02/why-i-love-coding/
Check out this guided tour of the 13 jhanas - https://www.youtube.com/watch?v=PRdiOoTZC3A
Am I the only one who believes that it is bad for me to spend more than 20 hours -- and perhaps as little as 15 hours -- a week in the state?
Just because an activity feels really good does not mean it is good for you.
I guess I should explain a little how I came to believe that that too much intense absorption is bad for me. Let me start by quoting a passage (which I agree with) from another comment here:
>in order to achieve flow these three criteria must be met: 1. The goal must be clear 2. Feedback must be immediate 3. There must be a feeling that our capability is balanced to the challenge at hand.
The most important things for me to have made progress on in my life are things where it was not clear what my short-term goals should have been (although of course it was important for me to choose some short-term goals to work towards), where feedback was slow and rare and where I felt that the challenges I needed to surmount exceeded my capability. (I would guess that there are problems or concerns like in most people's lives.) In my past, I spent a great deal of time using the pleasures of intense absorption to distract myself from more important things.
Even after I realized that I was using programming and related activities like math and "programming-languages geekery" to distract myself from more important (but less enjoyable and messier) things, I did not know how to stop the distracting activities. In other words, I was addicted to the pleasures of intense absorption, and it took me many, many years to figure out how to extricate myself from that addiction.
If there are a lot of things in your life that are pleasurable -- hanging out with friends, sex or romance, physical exercise -- then maybe you do not have to worry about falling into the same trap as I did. I think that probably what really tripped me up when I was a teenager and a young man was that intense absorption was my only reliable or regular source of significant amounts of pleasure. Well, to be more accurate, satisfying my curiosity, a.k.a., learning, and intense focus on programming and related things were my only reliable sources of pleasure. (I have read that this pattern of only being able to take pleasure in one or 2 things and the consequent problem of addiction to those things is often a trap fallen into by people who were abused as children. I was abused as a child. Perhaps people who had happy childhoods are immune to the negative effects of spending one's days intensely focused.)
I still program for a few hours every week: I write Emacs Lisp code whose only user is me, which I do not get paid to write, and I am convinced it is not harmful to me. It makes me happy when I notice that one of my problems can be solved with code because coding is an efficient way to solve a problem. But if I ever take a job that involves programming full-time, I'll keep a sharp eye out for signs of a recurrence of my previous destructive / addictive relationship with the pleasure that comes from intensely focusing or concentrating. A big warning sign would be if I were to start again to neglect things that clearly should not be neglected like appointments at the dentist and such. Another (more ambiguous) warning sign would be a cessation of the process which has been going on for over 10 years now of my slowing increasing the range or variety of things I am able to enjoy or take pleasure in.
I decided to write this because in online conversations among programmers, we almost never hear (or read) about any negative effects of the "flow state". For me, there were significant negative effects: namely, I would have learned to deal with messy, low-feedback difficult situations at a younger age if I had not spent as much time in the "flow state". In other words, I got into the bad habit of using the flow state to avoid what I really needed to learn: how to make progress in messy "non-flow" situations.
tldr You made me realize stuff from your post and how do you live without flow and also this is more or less for me trying to figure what the hell my problem is and why I'm not good at a lot of aspects that make a successful person, also you don't have to read it because its probably an unrelated issue that sounds like mine but its worth mentioning just in case it is and I rarely find anyone that I can relate to with this sort of stuff so forgive me.. just trying to learn more about myself.
When I start getting to the stage you are at, I make a point of doing some outdoor sports (a few people mentioned climbing. My personal preference is whitewater kayaking, or mountain biking - something that requires full focus and involves a bit of adrenaline).
I used to do a lot of outdoor, sports (worked as a river guide for a few years, and kayaked nearly every day). Now I have an office job, and see the difference. Something that gives you a bit of adrenaline helps you achieve the same feeling, as others have described with climbing, but not associated with coding.I think this helps you bring more balance to your life. If you don't do a lot of exercise, just getting fit will make a big difference as well.
At first it might be difficult to motivate yourself, but once you get into a habit you will find it easy, and you may even get annoyed the days you don't get to do your sport.
Anyway, that's what I find helps me. (Promising myself to start riding my bike on the trails on my way home from, now the weather is getting better).
That is misleading because it gives the impression that I spend a non-negligible fraction of my time in the flow state. I do not. I spend at most 15 to 20 hours a week programming. These years, very little of that time is spent in the flow state.
I take the position that a slow compiler exacts a cost far beyond the obvious, because it inhibits flow state.
The ahhhhh is like no else.