Programmer Interrupted
blog.ninlabs.com
blog.ninlabs.com
Each week, one pair of programmers would be designated the "consulting developers", and a big sign would be put above their desk. They were the only developers that could be interrupted for the week, allowing the rest of us to get a lot of work done. If the consulting developers needed to ask something of other developers, we tried to save it up for lunch, as we mostly all ate together anyway.
This made an enormous difference to our productivity, which everyone in the company took notice of when the number of "development days" we got done each week increased dramatically.
At the start we thought of "sacrifice one for the good of all" and we didn't look forward to our turn. As time went on it actually turned out differently. We usually enjoyed the "consulting" time as it meant a break from the routine of working on endless tickets, and it also kept us in touch with what the rest of the company was doing with regards to deploys, environments, configs, etc. etc.
AFAIK they still do it today
People are very respectful of the shared space and keep the noise down. Also, it's nice when you are not consulting to actively ignore whatever is being said to the consultants, so you can be more in the zone.
> I don't think there is any cure for interruptions other than no at-desk meeting rules or private offices.
That's what we do at my current company (try to limit drive-bys) and it's horrible - I just get people booking time in my calendar every 20 mins, so that it's not a drive-by when they want to interrupt me.
A simple feedback mechanism for this would get us quite far already. Just something that will glow red for 'dont disturb,' orange for 'only if you have to' and green for 'go right ahead.'
If someone wants to make (a Kickstarter for) a simple ball that does this I'll gladly transfer my domain names interruptiball.com and interruptaball.com to you by the way. ;)
Might work on this some in the near future, but I am not sure if I want to commit to a Kickstarter ;-)
Do you think it would be feasible to have a competent manager assign somewhat high level tasks to engineer-managers, have them assign subtasks to engineers, and then have the engineer-managers deal with integration, systems design, and interfacing with real management?
It seems like if that situation works in practice (I'm too new to know, and didn't go to management school, so this is largely experimental or anecdotal for me), tasks can simply be assigned by someone close enough to understand the complexity and skill of the engineers actually doing the coding via something like exchange based tasks, a to-do list of open issues that need to be dealt with, etc.
Or do you think that the core engineer staff is not mature enough to handle that level of independence at most big companies? I would assume that at startups the structure is more flat in such a way as to make my suggestion less meaningful.
I ask this because I'd really like to be a better manager, and I find that for myself, having a list of goals (i.e. "define API and write spec", "draw out abstraction structure", "CAD up mechanical enclosure", etc) is more than sufficient for me so long as I'm in regular enough communication with the people around me to know what those shortened descriptions mean for sure.
We're actually struggling with finding the right amount of time for this. A week didn't feel long enough for us because that person would also handle tier "3" support for the week. Any thoughts on finding the right amount of time for something like this?
Edit: there's also one person who is the permanent point of contact for a single release. New releases are once per three months. This person obviously has one hot week, but the amount of contacts decays exponentially.
What I've found is that the typical developer/engineer is really bad at saying "no" or "not now" to others. The solution then is to work the other side (sales, marketing, etc) to interrupt someone else (me) instead. Even if it's a relatively simple technical question from sales, I can convert the in-person active interruption to a passive email notice (at the very least).
The typical office environment (and system) is not productive, so at this point it's up to the individuals to reduce the impact of the system on the team's productivity. (which is a sad reality...)
The downside is that an assigned mentor will not get as much work done during that time.
Hopefully getting the interruption onto the schedule would have the benefit of allowing everyone to plan for it and try to minimize its cost. And it might also encourage junior members to try and work through problems themselves first (because they have to save questions for scheduled question times), but also make them feel less like an imposition when they do seek advice.
- no flak
- no flamewars
- no jokes
- no off-topic stuff
If everybody sticks to those rules then it will work fantastic, if you don't guide that it will turn into a zoo and become huge distraction.
We use a plain old IRC server + clients for this, nothing fancy.
The etiquette thing is important, and varies team by team. Our team has several channels to ourselves with varying levels of formality, and company wide channels similarly (#techteam, title line: "No banter" vs. #companyname, title line: <insert latest in-joke>)
Not only does the group element allow anyone who's taking a quick break to offer some help, it often draws in advice from across the organisation, including people you'd never have thought of asking, who just happen to be dipping in to the channel at the time.
Edit: The office wide "banter" channel's primary purpose most days (so like, not Friday afternoon) is the distribution of fresh coffee, prepared in 5 mug cafetieres, announced and claimed here. So many people have IRC bings on the word "coffee" that it's best discussed in some form of leet substitution if you're not offering to make some.
I'm more than a little jaded at this point, and would love to hear if anyone has faced and solved these sorts of issues.
The reason for this, by the way, is roughly that it's short-term easier to think about how to solve their problem so that they'll go away rather than "waste time" arguing about priority. People outside the engineering department tend to be a lot more well-versed in making a case for priority and instinctively push more than the engineer pushes back.
Also, many thanks for being one of the awesome people who help deal with this. As a developer who has worked under a few great manager-types (and has a ton of trouble saying no to a request), you make all the difference.
What's kind of funny is that while I'm pretty good at being a shit-umbrella for my team, I've found that I really really hate making urgent requests to other teams (HW, Applications, etc.) precisely because I know that they also have a lot on their plate already. In fact, I think I'm honestly really bad at 'pressuring' people in other teams to do things that our team needs. Everything comes with a cost, I guess.
> People outside the engineering department tend to be a lot more well-versed in making a case for priority and instinctively push more than the engineer pushes back.
I've also noticed that a lot of times, an individual sales/mktg/bizdev person actually doesn't know what the overall priorities are, since she is only aware of her customers and her items that need to be addressed. The person rarely has a sense for what the relative importance of her items are compared to those of the person sitting right next to her also asking the dev team for their time.
In a similar vein, I think it's very hard to push back as an engineer since she will rarely know all the projects that are going on concurrently, who the customers are, what the estimated revenues are for each project, etc. This is exactly the kind of information you need to fight against an interruption request: "Sorry but I'm working on item X for customer Y, with deadline Z, which is a higher priority item compared to your item W."
Now, an engineer probably shouldn't even have all this information, since the information is kind of a distraction in itself. So when a rogue customer-facing guy comes up to an engineer with a request, a much easier solution is to answer with, "Sorry I don't have the authority to change what I'm working on. Can you please go talk to my engineering manager A or my project manager B? I'll be happy to change course once I get updated instructions from them. Thanks."
I've also heard managers chewing out or giving educative talks to others in order to remind them that the correct person to make the request to is the manager, not the engineer.
Haha, funny you say this, since I also 'educate' others about this on a monthly basis (it's become much better compared to 12 months ago though, so credit to the customer facing teams who are generally reasonable people).
For me what ended up working by far the best is to always and for every request try to:
1) Give an honest summary of the negative consequences of your request being delayed/ignored
2) Ask for an honest summary of the negative consequences of your request being honored.
3) Continue calibrating the relative importance until the higher priority task is obvious for both parties.
At first you might get run over a few times but if you don't accept politics (also not your own!) to enter the equation and consistently do this eventually people will be very accepting and these request can be made very efficiently.
If his sources are outside of the company or also not truthful then I'd also be at a loss :)
http://blog.fogcreek.com/the-price-of-dev-happiness-part-two... http://www.joelonsoftware.com/items/2006/07/30.html http://www.joelonsoftware.com/articles/fieldguidetodeveloper...
I guess it depends on the kind of programming you're doing and the teams composition.
Yes, face-to-face, voice-based collaboration can be a good thing, but very rarely do you need it 100% of the time. Need someone to look at your code? Ask them to come over and look at it. Need to chat with someone to discuss the design of some code? You can still do that.
Why folks somehow tolerate "everyone gets bothered by any side conversation" is beyond me, especially when you can have perfectly fine collaboration without open-plan offices.
It's incomprehensible why companies with "communication problems" think that will be solved by throwing everyone into one room. I bet good money the problem isn't that Jane isn't overhearing Joe and Bob's conversation. It's some deeper disfunction that is being exposed as "communication problems".
Try as we might, some of us just can't do this.
The quiet environment I was describing can be found in an office where the majority of people are heads-down working and not on the phone. This is harder to find when you're in an office that's setup as a cube-farm or an open layout. But this can be easily found at the beginning or end of the day and especially on work days that bookend holidays or three day weekends - days that lots of people typically take vacation. Those are the times when I find I get my best work done because it is the closest to quiet and distraction-free that one can find, other than working remotely where you can control the environment.
As far as the "go-to" person people have noted having above, everyone on my team works on different aspects of what we do, so no one person can answer every question. Our PM is also not super familiar with what those roles are nor are they technically competent enough to answer those questions on their own (as of now, anyway).
While it might sound like the difference between introverted and extroverted, I suspect it isn't quite as simple. It very likely also depends on the job to be done (memory load), the content and quality of surrounding conversation and more ephemeral factors like company culture.
I suspect it also affects how well things like pair programming work out, conversations about it often elicit quite polarized viewpoints on the subject.
I think there's some level of cognitive bias here with respect to social interaction preferences.
I certainly prefer to hang out with folks in a social setting compared to sitting in an office all day. I understand folks who prefer to do this in a work environment from a camaraderie and a "I'm not in a cave" perspective. That's totally fine and understandable.
But from a real, practical standpoint - are folks that "prefer" to be in an open setting actually more productive than those not?
What type of programming lends itself better to a distraction-filled environment?
How does "culture" fit into the distraction equation, other than "we tolerate more distraction-killing productivity here than elsewhere"?
(We have a chat room where we try and put all side discussions, so folks can virtually "overhear" conversations and chime in if applicable).
I concede that there could be some people who will get so antsy in an office all day that the mosh pit-style work environment "feels" better. I'm not discounting that some folks prefer to be in a noisy environment. I just question whether or not they're actually more productive there or that the organization is better off in total with a noisy environment.
The difficulty I have is that I cannot find any studies that show that a distraction-filled environment creates more knowledge-worker productivity than a quiet one. Mountains of studies scream the opposite.
Personally, I'd rather find other ways to channel that ADHD energy in a more productive manner. :D
Though I doubt that it would be much different for me, personally, because when I'm really in the middle of an interesting problem and cranking stuff out, I just don't notice background noise in the same way I don't notice that it's suddenly four hours later... I mean, maybe I'm just lucky in that, but my own coworkers have never expressed frustration with the environment either, so I don't know just how rare it is. (So I guess the key thing is that if the environment is getting to you, don't just keep it to yourself!)
I also have a (maybe overly cynical?) hunch that in addition to the "ease of distraction by external noises" spectrum there's also a "desire to goof off more if nobody's going to notice" spectrum that comes into play. (He says, as he posts on HN in the middle of the day, heh...)
I've never bothered to look into research on it, but I also don't see any trends of companies with private offices outcompeting ones without. I know the output of the team I was on at the company I worked at where I had a private office was not great, but there are other factors involved there too, of course.
I also think the rotating "consulting developers" idea from grecy above sounds like a pretty good idea, regardless of office layout.
However, the important thing to consider is the fact that humans in general prefer to be among others. Even sitting quietly staring at a wall is more preferable in company than alone. So I don't really care if I'd be 10% less productive, being amongst others is good for my mental health.
However, this should of course mean that for those who don't enjoy company of others there should be a way to be alone. Just having a "silent room" where you can go to not be disturbed might be a way.
I vastly prefer open office where developers work in frequently rotated pairs, with separate pairs close enough to talk as necessary. It's a little bit harder this way, but I've found that it leads to higher quality software, higher diversity of knowledge and it's just made me a better programmer (and a better Vim user and a better Unix command line user, etc). I hope I don't ever have to take another job where I'm not forced to both teach and learn from all of my coworkers directly.
(Disclaimer. The broader applicability, appropriateness, and overall desirability of pair programming, Extreme Programming, and other agile methodologies, or any of the other implications thereof, are not addressed in this post. Void where prohibited. No warranty, express or implied.)
But. In my experience, when I was pair programming, and the pair programming was effective to begin with, then: when there were interruptions, the interruption-recovery process was significantly streamlined. I infer that this is one of the ways through which teams that actually already do pair programming manage to improve their productivity enough (relative to singleton-programming) to justify the arrangement.
Imagine something like the ubiquitous sleep trackers for mobile devices which can track your sleep pattern, and both give you feedback on how you sleep, and also select the best time to wake you up within a certain boundary.
Could we imagine something similar for engineers? We already have very rough tools to provide feedback on productivity, like RescueTime, but a context-aware tool that can model cognitive activity based on edits, switching from one function to another, viewing two files simultaneously, git commits, etc, is far more advanced.
Perhaps if you wanted to ask a programmer a question, you could ask for him to be interrupted sometime during the next half hour, and the system would choose the time where it would be least disruptive?
And I'm also interested in the discussion of ways in which it would be easier to remember/reestablish context. I wonder if some of this is transferable to other domains, for example deep academic "knowledge work" - working with annotated articles, several windows of notes, trying to synthesize ideas etc.
Of course, one could argue that the monotony of writing tests is what keeps me from being fully engaged to begin with :)
At least one of the ops people on my team and at least one of the devs on the other team were, well, loud. I don't know if they had hearing issues or what, but they were significantly louder than most people. You know the type. That's just how they are.
Immediately over the cube-wall from us on the other side was a bunch of technical writers. They couldn't get any work done because of the racket from us. We got complaints. My boss used to keep a tally on one of his whiteboards. This is the thing where you go |||| and then put a slash across for the fifth for those who aren't familiar (is this a local culture thing?). I think they ultimately tried to get moved to another building, away from the people they supported.
On a third side, there was a bunch of project managers. This was back in 2007, so OpenSocial was the "big thing" then. So we had to deal with the blathering coming out of them most afternoons: "what if this person is friends with this person on this service, but isn't friends with this person on this other service, and then this third person shares this thing to the second person, does the first person get to see it?"
I'm surprised ANYTHING got done there. Come to think of it, that might explain a few things. Another team up in that same space was the Lively group, and everyone knows what happened with that project.
2) Many tasks (especially engineering that deals with work in the physical domain) require tools, items, equipment of more than just a computer. While we can do shared lab space for development - it does get tricky when you need certain hardware or test setups to do design/work.
It did take me time to adjust outside of academia too.
Daily scrums should not take longer than 15 mins. A good scrummaster will keep them under 5 mins in average for a 12 member team.
One thing I usually do when daily scrums start getting out of hand is to politely mention long meetings as a blocker to get my job done. After repeating the same for a few days it will start to sink in...
Instead of trying to fight the interruptions themselves, I decided to try and become better/faster at context switching by training myself to compartmentalize the jobs on hand (programming/helping kids/answering questions).
It took a while to figure it out and i'm still getting better at it, but so far it works pretty well and I can jump between contexts much better than i could before. When I get interrupted, I'll take a snapshot of where i am, store it away and 'switch' to the new issue.
This is one of the reasons why agile methods work so well. Small tasks limits the scope of concentrated work, reducing the risk of going astray. Pair programming offers a watching party, who can guide and direct while you are busy tunnel-visioning some small method.
The best, "snake-oil," solutions I have are:
1. Well-delineated boundaries for interlocutors. This means scheduling my, "off-limits," time and communicating those boundaries to the people around me. Concessions are made by appointment and I try to plan out my time a few days to a week in advance depending on how busy I am.
2. Write everything down. I try to be methodical about this but I am characteristically spontaneous and disorganized. Therefore I keep notebooks nearby when I'm not at the computer, I have a whiteboard near my computer, and I have a very organized filing system on my computer where all of my thoughts eventually end up. I find that having different mediums that I can switch between very helpful -- many an algorithm has been conceived of in the shower, doodled on a piece of graph paper (I'm a very visual thinker) over breakfast, and eventually compiled before lunch after I've finished with my emails for the day. Having record of it in multiple places seems to reinforce my memory and after reading this article I suspect I now have an inkling as to why.
3. Never leave a thought unfinished. This can be frustrating both for myself and my spouse sometimes when it's near the end of the day and I'm only half-way through something important. However I cannot enjoy myself and let my mind wander if I have some half-finished thought still rummaging around in my head. It's easier to let the mind relax and be creative if you can finish each thought... even if it's simply writing the rest of it out or hammering out all of the test cases you can think of. As long as there is some record of it in its entirety my mind seems to be able to be satisfied with it and leave it alone. It also helps so that when I get back to work on the problem that I'm able to get up to speed and not having to, "finish my own sentences."
After reading this article I began to wonder if there were any cognition-enhancing tools that are integrated with popular development environments for assisting programmers in remembering where they were in a large refactoring or what they still have yet to do in various locations of a code-base, etc.
I found org-mode to be great in tracking todo items, general note taking, and organizing thought.
"A code narrative is an episodic memory aid that helps a developer recall contextual details and the history of programming activity."
I created Labradoc (http://labradoc.com/) to encourage a form of manual "code narrative" (e.g. a form of development journal).
Here's an example from one of my personal projects: http://www.labradoc.com/i/follower/p/android-arduino-handbag
I find it really valuable for context-switching between projects.
[1] http://tomato.es
I find it takes me much longer to get back on track after interruptions. I work from home and deal with a ridiculous amount of interruptions. After 9am, I usually have to just give up for the day.