Holding a program in one's head

Writing software is kind of the same way. Just look at the impossible made possible by demo coders on old-school 8-bit computers. These programs would never have been successfully coded in a commercial organization. But, -bands- of coders wrote them successfully. They promoted their software as bands, included self-written music, artwork, etc.

It would be interesting to see how a band-style organization would apply towards more practical software products. Software so produced would come in boxes with the band's logo, but more importantly, a _list of credits_, anecdotes about the software's creation, etc. That is, to make the delivery of the software more _human_.

Back in the day, when credits on software were more commonplace, it was possible to judge the quality of a product (to some extent) based on who was involved with it. Some people became reknowned coders, reknowned technical writers, etc. I think it gave two incentives: first, your name is going on the box of that package -- this gave prestige in the community post-sale; second, it allowed the customers to predict the overall feel of the software prior to actually purchasing it, based on their experiences with software written by the same or similar authors.

Literature is another example. People flock to this blog because of the name, Paul Graham, just as much as they do for the information contained therein. People buy books from famous authors because the authors are well known to produce good work. People often subscribe to magazines only to read one or two columns by well-known authors. So, in a very real sense, tacking your name on something is a seal of authenticity and a seal of quality all rolled into one. And, people like that.

I know I do.

God, how I wish more people understood this. It might seem like programmers overreact to even the most minor demands on their time---"What's the big deal? It's only a half-hour meeting!"---but a half-hour meeting can easily kill several hours of productivity.

A good friend of mine is a artist. He claims his secret is, "I paint every day." I tried that. It didn't work for me.

I have tried every combination of pens, tablets, paper, sticky notes, and electronic approach to distill my thinking and none of it has ever made much difference.

Only when I get the entire program in my head (level 0 only), do things get cooking. I rewrite everything 3 or 4 times. Sometimes I rewrite just to understand.

This approach reminds me of Jessica Livingston's chapter on Steve Wozniak in "Founders at Work" (required reading). Only when he could get the entire Apple II into his head did it become the breakthrough that it was.

I used to be afraid to exercise because it took time away from programming. What a mistake. I walk up and down steps for 35 minutes every morning WITH MY PROGRAM LOADED INTO MY HEAD. THAT'S when I do my best design work. I didn't realize what was happening until just now. Thank you!

What I think is more likely is that you get fresh insight when returning to a problem after a break, because the mental model that was constructed in your head was subtly wrong. It's happened to me a bunch of times: "OK, where was I?... the grommit plugs into the foobar, and the whatzit sends a message to the widget... wait a minute, no it doesn't, it sends to the frobnosticator first! WHY on earth was I thinking that for all that time before the break??"

Sometimes it's as simple as the famous case of getting somebody else to look at your code and immediately point out what's wrong that two hours of staring at the screen couldn't accomplish.

Other times it's a lot more subtle, high-level and abstract. Since, in those cases, it's often difficult to discuss the problem with somebody else, you don't get a) the outside feedback to speed things along and b) the humiliation factor that makes you remember how the insight came about. ;) So the meta-revelation of how you came to that revelation might be lacking, and it's attributed to the subconscious instead.

Interestingly, as I headed to grad school, this became more and more difficult as the problems became harder and harder. Eventually I had to devise a new system for writing papers (which I can't describe adequitely) because the problems and their solutions became to large to hold in my head at once.

To me, this is the interesting case, how do you solve problems that are too large to hold the solutions to in your head at once? The obvious answer is "break it into smaller pieces", but that is frequently very difficult to do. Probably the answer lies in "go back to the basics", and make sure that you have the foundations cold so that they aren't occupying stack space.

If I work on several project concurrently, there's always one of them that I am more passionate about at any given time, causing me to think about it at nearly all hours of the day. Interestingly, the one I am more passionate about oscillates -- and is nearly always the project I have most recently worked on for a large chunk of time. Therefore, it is the one in which I am holding the most context in my head.

This happens to me on so may occasions in may different areas. While programming, if I have a problem I'll sleep on it, wake up, and see the problem in a whole new light.

Also, before writing an essay for class, I will read the prompt throughly. This way my brain starts form sentences without conscious thought.

Naur, P. 1985. Programming as theory building. Microprocessing and Microprogramming 15, 5, 253--261. http://www.zafar.se/bkz/Articles/NaurProgrammingTheory

The only part I don't really agree with is the implicit condemnation of programming done by large companies. Yes, their methods result in mediocre software, but that's often what you want. There's a reason they try to treat programmers as interchangeable cogs, and resist having an entire program in one person's head.

To use the tired "building a house" metaphor - you can get a renowned architect to design the next landmark in a city, or you can get mediocre, interchangeable architects to design a row of townhouses. Both approaches are valid and have their own place, but there's no point asking the famous architect to build townhouses.

Psychologists have identified a state of mind called flow in which we're capable of incredible concentration and productivity. The importance of flow to programming has been recognized for nearly two decades since it was discussed in the classic book about human factors in programming Peopleware: Productive Projects and Teams by Tom DeMarco and Timothy Lister (Dorset House, 1987). The two key facts about flow are that it takes around 15 minutes to get into a state of flow and that even brief interruptions can break you right out of it, requiring another 15-minute immersion to reenter. DeMarco and Lister, like most subsequent authors, concerned themselves mostly with flow-destroying interruptions such as ringing telephones and inopportune visits from the boss. Less frequently considered but probably just as important to programmers are the interruptions caused by our tools. Languages that require, for instance, a lengthy compilation before you can try your latest code can be just as inimical to flow as a noisy phone or a nosy boss. So, one way to look at Lisp is as a language designed to keep you in a state of flow.

PG, you got the Stephen Wolfram to read this? If so, perhaps you can have a conversation with him about making Mathematica open source :~)?

By Paul's account, the different components of ViaWeb were all written in different languages. That enforces vertical silos to a degree that I'm not sure would be healthy in many projects. Just yesterday I saved my colleague a lot of time by pointing out that he was basically recreating a library function I'd already done.

Projects that have clear vertical components, and one team member per component, do move really quickly and it's tempting to think that all projects should work like that. I'm not sure that is really true of all worthwhile projects. However, it might be true for startups.

Multitasking is a myth. Editing code on demand is possible but to design a program is a process that requires all cylinders of the analytic and creative mind. To be in the zone is almost like a trance where I could start speaking tongues at any moment.

More often than not, the reload is imperfect and parts of the universe are not restored, leading to more lost productivity as the programmer must recreate the solution they already had worked out. Another subtle side-effect of this imperfect reload are bugs caused by a mismatch between the pre and post-interruption program state.

"I suspect there is an issue with the actual input from the database. (Note: we're working with a test database that's sketchy in spots) What do you want me to work on? Do you want me to spend the time tracking down that error or do you want me to work on "X" (that should have been done three weeks previous)? Because I was thinking about "X" and my entire train of thought is derailed and now I'm trying to work on "X" and wondering what is wrong with that particular chunk of data."

I suppose for the non-programmer it sounds like I'm just being nasty over nothing, but that little panic attack over a minor error message cost me HOURS in trying to get back to the original program in my head so I could finish it. In fact, I actually had to go and FIX the error to get it the hell out so I could fully focus on what I needed to be doing.

Once I got there, around 1am, the code flowed like water and it's done save for minor debugging. I was in the zone enough that if my eyes hadn't been closing by themselves, I'dve finished that, too.

Management, I think, has a double edged sword to deal with. My immediate boss wants to let me do what I do because I am lucky enough to have the ability to put someone else's code in my head in the same way described in this blog entry. It means that I can go in and fix it and if there's a bug I know why it's doing it and I know just where to push on it and where it needs shoring up--in short, after a while, it's like I wrote it myself.

But he's also dealing with HIS boss, who is dealing with the bottom line, and his boss would probably have apoplexy if he saw me playing spider solitaire as I let the problem I'm addressing work itself into my brain, and would have NO idea what I was talking about if I said I needed to get the application "into my head".

Wonderful blog entry.

At my old company, one comment you'd always hear from team members with a new idea was "Hey, I thought of this in the shower this morning, and I think we should ..."

Holding as much of the program in your head gives you the power to identify what will give you the best optimization. Also this may be the short explanation why good software is written by teams of max 2-3 people!

There is also another issue that seems to be overlooked, people create their own disruption. When I wrote about Gloria Mark's finding on working spheres ( http://nuit-blanche.blogspot.com/2006/07/designing-collabora... ) I was surprised of that finding as it would seem counterproductive to the need to having to load up the "code" in the brain's RAM. Another explanation is that what she described is a typical cubicle/large organization workflow that would be counterproduictive to the Hacker's brain (replace hacker by researcher and you have the same symptoms).

Igor.

Holding the program in your head "scales" - at least on well-constituted teams. I was a chief/architect (based on Fred Brooks' chief programmer team concept, long time ago) and was able to utilize some "ordinary" programmers and a couple gifted programmers by doing the highest level abstracting of the problem so they could each hold a sub-problem in their heads. Made the bosses happy, since there was less reliance on any one person, and I was largely replacable by one of the more gifted team members.

(BTW - greetings from anotehr Gateway-survivor)

I've been trying to get into a small program I need to write and am a bit stuck. After glancing at the (great) list in the article and realizing I can roughly control most of those elements, I think 'now why am I still stuck'? Oh I need to get this data structure from X, how to call that sub element from Y. Grunt work, yet details that easily fit into gaps between all the other various day to day distractions. So get all the junk together before that 12-36 hour marathon.

In my opinion, there have been only two significant developments in mainstream languages in the last 50 years - structured programming (if..then..else) and object orientation. The operative words are MAINSTREAM and LANGUAGES, as there have been some very innovative and terse non-mainstream languages.

In the beginning, there was FORTRAN. You forced your problem to fit in a series of matrices and got the job done, usually quite efficiently. Add to Fortran structured concepts, and sooner or later you get Pascal, PL/I, and C. C introduced pointers - for better or worse, programmers finally had time to get really creative in solving problems by inventing all sorts of data structures - some legitimate, some a distraction from the problem. Bigger code. Add to this rampant object orientation, one method per class, and the ability to really go overboard with content vs. form. Form is winning. Bigger programs.

Although not part of a language, these days with Java, .Net, and so on, it is not possible to get by without a heavy duty IDE - Eclipse and Visual Stupido. These tools help you navigate a problem which you can no longer possibly load in your head, hopefully you can get a little part of it.

To me, the place where there has been the most progress, and outside of programming language design proper, is in the field of memory management and garbage collection.

Maybe everything I know is wrong. But in the last 20 or so years, I have never seen a programming language which stressed clarity through conciseness. It's all starting to be more like COBOL. Is this what the world really likes?

The idea here is that organizations are inherently anti-individual. My essay takes the idea that organizations are anti-individual because of their history. I think there is some merit in both points of view.

I disagree with rewriting one's program. If I rewrite a program to solve the same problem, then I'll end up having exactly the same code. This is useless. If I am solving a different problem, then rewriting is usually in order.

In regards to writing (re)readable code, I feel that the word "readable" leaves the door open to a lot of abuse, and doesn't shut the door firmly on those who encourage literate code, James Joycian code, commented code, etc! "Readable" has connotations that one can read it at leisure, like a novel. It allows for some excess, some baggage. If there were an adjective which meant "the uncompromising naked terseness of mathematics, complete absence of comments except in pathological cases (to be avoided!), and short identifiers, especially in inner scopes," then I'd use that adjective.

I usually think in terms of Occam's Razor: what minimal program description reproduces the subjective effect that a user desires? I feel that any other approach would be based on ideology. "X-Acto blade like code," perhaps, rather than "readable code?"

1. The leader of 130 IT workers was not educated in IT. None of the top leaders were into IT. Not even the lead architect!

2. The economists think of everything as a "factory". Therefore, IT is produced the same way. I call this the "factory-view". If you add more money, more developers you will get this done faster. If you need something, go buy it. It's always better to outsource.

There were other things as well, but these factors may probably be more widespread than I like to think of. Finally I left for a small firm. What a relief!!

On a seperate tangent, Paul referred to some mental techniques (e.g. 'black box', 'solve a subset) and I can think of others (e.g. hold this portion 'constant' and change a different portion).

What if we had a catalog of those mental techniques - would that be interesting? Usefull? Who would use it?

Thirdly - my experience of loading the whole program in my head extends beyond programming - when I've participated in the business side, along with programming, then the scope of what I consider becomes larger (and involves some new elements) but the underlying approach and way of thinking about problems/solutions appears to be the same. Have other folks had a similar experience?

Companies that allow interruptions or distractions that don't let employees get into "flow" or "the zone" and create/execute at their capacity are going to find that it's difficult to retain truly great employees because most great creators/makers are only happy when doing their best work - anything less is demeaning, insulting, boring and maddening.

The points made by other comments that some companies fear the gifted individual because that individual is not a cog that can be exchanged for others is correct - I've seen this fear before. So what's the answer? Have some of both kinds of coders? Accept a loss of control and the associated risks? Have an Advanced Technology Group (ATG, like apple once did) and let magic happen there and then toss it to the production group to put it into shippable products? How do you keep the envy between those in the group and the rest of the engineers under control? One thought was to have rotating positions in the group and you did some time in production and then you rotated into the ATG. I like the sound of the Google 20%, but don't know how well it would work for me - I want a couple of weeks per idea to really get it to the prototype/proof-of-concept stage. Then I can set it aside.

At another company, I had the software engineers in a separate building across town from the sales, marketing, support groups. And everyone one who was not a programmer was directed to contact me and not to contact programmers directly unless the programmer had requested them to. Not ideal in all ways, but it did help with many of these issues.

anyway, truly great post. I sent it to the VP of engineering at my company because it explains my frustration with working there so well....

This topic is worth a book on how to run a software engineering company. Take the insights from this blog, and then figure out a way to create a medium to large company work environment that supports what software engineers need to be truly productive and innovative.

I don't think it's important for a program to have a small text-footprint in order for it to be easily loaded into your brain. Why? Because it's the CONCEPTS behind the application code we're loading into our heads. It's not the text itself. So... if I have a program where I've formulated a class tree, it doesn't matter to the ease with which I can grasp the program whether the classes are formulated in Python, C#, C++ or Pascal. It's the concept of the class tree which we load into our head. Not the code itself.

So, on that point you're actually wrong. Brevity isn't important.

That said, a lot of your other points actually DO make sense.

PS: IMO there is a difference in power and efficacy.

That's on sentence that resonates and is probably a clearer definition and advantage of what working on a "hard problem" is really about.

-sdk

Kujoy

When mentoring smart rookies, this is the bit some are last to grasp. Partitioning a problem means that there isn't as much to remember. This scales, fractal-like to the design of larger systems.

I find as I get older, this is more and more important and necessary.

The problem is the people who impact problems and haven't got a clue. There seem to be a lot of them and they are dragging mankind backwards, slowing and even reversing progress. These are the turkeys who blame programmers for a majority failure rate of projects.

It's important to deturkify these guys. What can you do to help that happen?

The only downside to working on your own is that you don't see the bugs that you create through not seeing the problem? Others seem to see what you don't? That solo work needs tempering with some level of review.

I chuckled over the effort spent on 'homers', work done (perhaps even for the company) in a persons own time. Boy does that work get some deep concentration and produce some good results!

Thanks Paul. DaveP

> Perhaps the optimal solution is for big companies not even to try to develop ideas in house, but simply to buy them.

What's good for YC is good for the country!

Although there's more inertia than working solo, I like team pair programming better. You get third-draft code in first-draft time when pairing, and having to continuously collaborate on the design sharpens your thinking a lot.

for me the circumstances for working on my theory must be similar to the cristalls that grow especially good at zero-gravity - the best place for me to think is in the train where i have 6 hours without being disturbed!!!

http://joelonsoftware.com/articles/fog0000000022.html

Programming is fun. Bug fixing is not. Selfish programmers, however talented, don't fix bugs.

Filtering out selfish talented programmers is not easy. Yet they probably are the reason why organizations get paranoid.

An extreme solution is to make sure bugs are fixed (somehow) BEFORE new code is created ; selfish programmers won't stand the heat very long.

Fortunately test suites, automatized non regression testing, is becoming popular.

Rajiv.