Some lesser-known truths about programming
dotmac.rationalmind.net
dotmac.rationalmind.net
"Although most software is made by teams, it is not a democratic activity. Usually, just one person is responsible for the design, and the rest of the team fills in the details."
When things are designed by 1 person (IME) they work better and have a more flowing work flow. Designing in a committee has been a freaking nightmare.
"Programming is hard work. It’s an intense mental activity. Good programmers think about their work 24/7. They write their most important code in the shower and in their dreams. Because the most important work is done away from a keyboard, software projects cannot be accelerated by spending more time in the office or adding more people to a project."
While I don't 'code in the shower', I do think about my projects constantly. An 'ah hah!' moment can come at any time. Just the other day I left for lunch and in the car on the way I thought of a possible reason for an issue we were experiencing. I was getting nothing while staring at it.
I completely agree. Yet in industry, being that one person (like I am) leads to great management consternation to the tune of "what if you're hit by the proverbial bus?" which leads to "you need an understudy", which leads to expending 2x the resources to achieve <2x results (the time spent selecting, training and guiding an understudy is definitely nowhere near 0, even when the choice of understudy turns out to have been a good one), and there can't help but be some feeling of "training your replacement". Industry loves interchangeable parts (people), and a one-man-band (domain expert or similar) grates against that ethos.
If a key employee, being the architect or even the CEO, is hit by a bus, the company will take a hit. If employees are a central, strategic assets, you can't go around killing them off and expecting no impact on your business. Deal with it.
The important metric is whether the company/project can recover, if it is resilient. Just because one guy drives the decisions, it doesn't mean that the team filling in the details don't know the code and the motivations behind it, and there can't be a guy from that team who could take a promotion, or that the team couldn't be reasonably productive for a while, while a new architect is brought on.
The bottleneck on many projects (especially those that are either poorly defined or particularly novel) is not coding speed, but rather learning speed. The faster the team learns about the problem it's really solving, the environment the solution has to live in, the people who will have to use the software, and what the possible solutions are, the faster you're done.
And that certainly matched my quarter century experience in the field, and boy was it frustrating to watch teams try to build frankly impossible systems.
Heck, I'd be happy if more people got and could use big O notation. In one particularly unhappy case my peers just couldn't grok that the design would vastly outstrip the database's maximum transaction rate in normal customer configurations (number of client workstations).
The author describes developers who spend much of their time writing code as "lazy, ignorant and arrogant". I think it is the author who is being lazy, ignorant and arrogant in thinking that he knows the truth about programming.
And the idea that a developer will only write a handful of lines of code each day that will end up in the final product might be true for a developer working on an established piece of software but anyone who has worked for a start-up or on their own software will just laugh at the idea of writing just 12 useful lines of code a day. A good developer can easily do hundreds, if not low thousands of lines of code in a day if pushed, and this code does often end up in the final product.
Plus, he seems to think that a great programmer is someone who connects together bits of code created by others, rather than writing his own code. That's ridiculous - is someone who uses another person's code a better programmer than the person who wrote the code he is using, in which case I am a much better programmer than John Resig, say, because I simply use jQuery rather than having written it myself.
I think this article is a perfect demonstration of the Dunning-Kruger effect. Sorry for being so blunt but the article bloody annoyed me.
It's also easy to think you are vary produce when you produce 10k lines of code in the first six weeks of a project, but the reality is over time most of that is going away. And writing more than 10 lines of code a day that sticks around for more than 2 years without making people wanting to replace it is hard.
PS: Google, for example, often scraps the source code of any company it acquires so the real metric of lines of code produced per day over the first six months is often zero in a successful project. Many a startup has failed and nothing of value was lost.
It may be a semantic argument, but basically if the same people / team are working on the same problem then a complete rewrite does not make a new project. As far as I know zero of Windows 3.1’s codebase made it into Windows 7. However, when you look at API’s it’s hard to call the Windows 3.1 to ME family a separate project from the Windows NT to Windows 7.
Ultimately, all products will be scrapped or superseded but to say that means every bit of code you have written is unused seems a bit odd.
A more reasonable metric might be, how much of the code you wrote today will still be in use in 15 years. And if you think more than 10 lines per day is going to make that hurdle, I think your fooling yourself.
Even so, a few thousand lines of a code a day is a bit much for anything non-trivial. To write that much code in a single day means that person must have done little else besides coding. Unless there was a period of about a week of design before that kind of coding, I'd expect the code to be rather kludgy because everything would be decided immediately. A few thousand lines of code, though, is definitely not something I'd consider average.
A /small/ project that added a new feature to an existing system took about 30 hours for the initial version over a two week time period and about 800 lines of code. Granted, I don't know exactly where my programming abilities lie skill-wise, but I can't imagine someone being 30x or 40x more productive than that as you suggest.
And maybe I'm just new around here, but if was going to argue for the Dunning-Kruger effect, I'd begin by arguing that I am clearly not suffering from the effect myself.
I think that perhaps there are two types of programmer - those who like to spend most of the time researching, drawing diagrams and the such like, and the others who just get on with it.
The suggestion of this article is that those who spend most of their time planning, rather than programming are somehow superior - my experience with developers has been the exact opposite.
Anyway, it triggered a classic defence mechanism: "what is calling me bad must be wrong". So he attacked the part of the article that offended him. In the process, he overshoot, and tried to discredit the whole article (by saying "complete bullshit").
(By the way, I only talk about the emotional process that likely triggered alexkearns' comment. That doesn't make it wrong. Nor right.)
But just to clarify a few points, I did not take the article as a personal attack on me or my programming philosophy.
I just happen to think that in its key points the articles was completely wrong. I know of no programmer - good or otherwise - who only produces 12 or so useful lines of code, or spends 80% of his time thinking about coding, rather than actually doing it.
Now, maybe my situation is not typical (but I bet it is not that atypical) but when someone claim's something to be true that in your experience is clearly not, it is one's duty to point that out.
Also, anyone who claims to know the truth about programming - I would certainly not claim I do - has to be dead certain they are right before putting these claims to publication, and one has every right to ridicule them if they get it wrong. Claiming to know the truth is not something that should be taken lightly.
To the point, I think you and the author disagree on some definitions. Most likely "writing code" and "lines of code that make it to the final product". Plus the unspoken assumptions about the project (small or large, early or late…), but that has already been pointed out.
I also don't think the article suggests that "a great programmer is someone who connects together bits of code created by others, rather than writing his own code". What It does suggest is that great programmers don't write their own code when they shouldn't. That when they do anyway, it's probably out of laziness (screw the searching, I prefer coding), or arrogance (I can do better anyway), or ignorance (what? It has been done before?).
You are right to ridicule those who utter bullshit while claiming it's universal truths. But be careful not to leave holes in your rebuttal. That include angry tone and amplifying the opponent's words.
So, it's either being condescending or not pointing out your mistakes at all. Do you really think you would have done otherwise? http://xkcd.com/386/
It feels like I am more and more able to see flaws in other's reasoning, but it's still harder to see them in mine. I also didn't figure out a way of saying "you're wrong" which doesn't hurt people yet. Finally, my current priority on the internet is seeking truth, not being nice.
So I guess that makes me an arrogant jerk. Well that's not cool, maybe I should shift my priorities a bit.
Why cannot we generalize?
I think most of the same conclusions apply to teachers, lawyers, salesmen, marketers, architects, physicians, nurses, engineers and scientists, whatever.
Wherever knowledge is valued in a job and there are intricacies lots of research, smart planning and thoughtful execution with a great commitment is going to make the worker a lot more productive.
Programming isn't necessarily unique any more than any of the fields you mention are unique.
"Why cannot we generalize?"
... but here's where you're stretching it a bit. I agree with you that programming isn't as unique as a lot of these blog posts make it out to be. But there's no ignoring the fact that different fields have different requirements.
For example, I don't think that a good programmer necessarily has the skills to be a good teacher (unless it's a computer science teacher, and even then it's debatable). Computer science involves translating logic into code. Teaching requires you to make logic suitable for people less educated than yourself to understand. That's not to say that a good programmer couldn't learn to teach or vice versa. It's just that they're orthogonal skill sets.
In college (for my CS degree) I had to take a technical writing class. I loved it. I knew next to nothing about the field when I started, but soon found the idea of translating complex ideas and concepts into more easily understood formats quite intriguing.
It was, in some ways, much like programming. I came to appreciate how hard it is for some people to understand some ideas (a point later reinforced when I started tutoring), and learned ways to improve descriptions and presentation.
It seemed to me that if you are involved in some geeky tech field it behooves you to learn how to communicate with Earthlings. Bizarrely, when it came time to pick a minor, my counselor made a comment to the effect that with a CS major I should be looking at, oh, a EE or ME minor sort of thing. Not something out in left field, such as tech writing. I hadn't yet told him that in fact that was the minor I was going to pursue. :)
Well worth the time and effort; if you have great ideas but cannot adequately express them, you're losing.
Engineer, scientist and programmers are probably in an unusual situation - they are exploring a primarily unknown solution space.
Most jobs live in the well explored, primarily understood areas of solution space, and consist of simply applying already known solutions repeatedly. Teachers, doctors (mostly) and most nurses just perform a set of rote tasks of varying complexity. The best way to improve performance in these fields is to write a script/checklist/decision tree and order everyone to follow it.
(In the case of doctors, I exclude a few at the very high end. Most doctors are not House.)
Engineering is applying the lessons of science to achieve predictable results.
Of course, there are engineers building yet another overpass/HVAC/etc just as there are programmers building yet another CRUD app, or scientists sequencing the DNA of yet another bacteria. And now that I think about it, they may make up the majority.
In that case, most pf programming is not so special at all.
This depends on what you are doing. Some programmers are exploring a primarily unknown solution space, and the best programmers probably self-select as this is difficult.
But the vast majority of programmers are busy applying the same well understood techniques to different problem domains.
E.W.Dijkstra, "The humble programmer", ACM Turing Lecture 1972
"Let me conclude. Automatic computers have now been with us for a quarter of a century. They have had a great impact on our society in their capacity of tools, but in that capacity their influence will be but a ripple on the surface of our culture, compared with the much more profound influence they will have in their capacity of intellectual challenge without precedent in the cultural history of mankind. Hierarchical systems seem to have the property that something considered as an undivided entity on one level, is considered as a composite object on the next lower level of greater detail; as a result the natural grain of space or time that is applicable at each level decreases by an order of magnitude when we shift our attention from one level to the next lower one. We understand walls in terms of bricks, bricks in terms of crystals, crystals in terms of molecules etc. As a result the number of levels that can be distinguished meaningfully in a hierarchical system is kind of proportional to the logarithm of the ratio between the largest and the smallest grain, and therefore, unless this ratio is very large, we cannot expect many levels. In computer programming our basic building block has an associated time grain of less than a microsecond, but our program may take hours of computation time. I do not know of any other technology covering a ratio of 1010 or more: the computer, by virtue of its fantastic speed, seems to be the first to provide us with an environment where highly hierarchical artefacts are both possible and necessary. This challenge, viz. the confrontation with the programming task, is so unique that this novel experience can teach us a lot about ourselves. It should deepen our understanding of the processes of design and creation, it should give us better control over the task of organizing our thoughts. If it did not do so, to my taste we should not deserve the computer at all!"
Edit: http://userweb.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD...
Often, people try to circumvent that problem by providing complete specs upfront. But those are doomed if they lack the old problem, i.e. if they were written by people who are missing expertise in either programming or in the domain.
Another way to circumvent the problem is to establish a good communication flow between the domain experts and the software developers. However, that only works when this intense communication eventually makes at least one programmer a domain expert. So that strategy is more a way to reach this state rather than to circumvent it.
Whenever a software project works really well, there'll bee another interesting phenomenon: The developers acquire a deeper, more abstract knowledge about the work practices than the workers themselves.
If there are people thinking they're bullshit, then the author is twice right.
You had me nodding along until I came across this. This single sentence is not just wrong, it's so wrong that it unfortunately calls the rest of the article into question.
Says who? Truths are not always good things. That doesn't make them any less factual.
This dovetails quite reasonably with Brooks' claim that we should plan to throw one away, because we will anyway.
It all totally depends on context, and these articles always use the context from which the author is biased.
Note that there's an interesting way in between. That is, refactoring code with the explicit goal of eventually replacing everything. There are several strategies to achieve that.
Those techniques proved to be less expensive and especially less risky than rewriting everything from scratch. They work because although the old code has an ugly design, it still has a property that is worth keeping: It is working and (mostly) debugged code. Undervaluing that property is the root cause of the many failed attempts to rewrite big software systems.
« Some lesser-known truths about programming » sounds better than « A few things that those of you who never read Dilbert don't know about programming with a team for a big company », i guess, but the latter is much more accurate.