On programmers, deadlines, and “Agile”

I'd never made the connection between "low priority project" and "strictness of arbitrary deadlines" as the project having to continually race against its' own insignificance. That's a fascinating insight and raises a number of questions.

For example, 'crunch' is something that's endemic in the games industry and a contributing factor is that big publishers will impose arbitrary deadlines on their studios to 'ship a game'. So, what does this say about the significance of these projects? Also, are the deadlines really useless, or do they have some real basis? (EA is not going to go broke because a Need For Speed game is a few months late, for example).

On the other subject of programmers being bad at estimation. I've done a hell of a lot of introspection with regard to this and why I find estimates so unpleasant. A few reasons that come to mind:

- When you're asked for an estimate, what you're often really being asked for is a deadline.

- In a sprint planning session, I'd often get asked to produce an estimate for "how long will it take to design this?" followed by an estimate of "so how long will it take to implement?". Which makes no sense - I can't answer the second question without answering the first. This often came up with bug fixes or 'small' jobs - things that could reasonably be assumed to fall within a single sprint for at least some of the time.

- I feel like there's an unspoken rule that your estimates relative to other team members signal your competency to the team and to your manager. So if you're constantly providing estimates that are longer than those made by other team members, it's a bit like you're saying "I'm less capable than them". You feel encouraged to provide short estimates and then hide the difference. I see this come up most starkly when sprint planning wants consensus estimates, despite having team members of varying skill levels in the particular task being estimated.

- I find that 'watching the clock' breaks my flow. If I'm having to track my own time, it's like a bit of cognitive load that I don't want. I don't want to have to think about it, because losing your sense of time is so crucial to really being in-flow. Often tools will ask you to enter how many hours you worked on something as part of tracking. So you end up even guesstimating at that ("well it felt like I spent half a day working on it, so I'll put in 4 hours").

- As a result of the previous point, I don't really collect historical data that I can reflect on. It'd be great to be able to go back and see how long it actually took me to do stuff. Not to mention that when you leave a company, you generally lose access to any historical performance information like that which you may have built up over the years.

There is a lot about Scrum that I like, but as a whole I agree it still suffers from the arbitrary deadlines problem. I wonder if there's a business acceptable way that you could do sprint-like activity, eschewing the idea of time estimation and replacing it with effort estimation only. Rather than aggressively fixing the time through schedule, aggressively fix the scope of work and then just work towards completion "until it's done".