The software development final exam: Computer Architecture and Operating Systems
daemonology.net
daemonology.net
Also note that you will come across people like this quite a lot in the software industry and you have to be careful not to let them get under your skin. They can be very difficult to work with. If you are a team lead and identify someone like this on your team then it can be very difficult to place them and you often have to gently coerce them into their own "special" area where they feel their talents are best used.
Don't you think that is a bit harsh? From what I have seen of Colin's work and contributions here on HN he seems like a pretty knowledgeable and decent chap to me. I rather enjoyed myself contemplating these questions - although I haven't had the nerve to submit completed answers!
This assumes an air of authority that the poster clearly cannot posses; that is, he does not have the right to judge whether someone is a valid software developer or not. Someone claiming this level of superiority seems to fit the personality type I described. Of course - I could well be wrong but that's what it sounded like to me.
"If you can't answer the majority of the questions on these four papers, and you're working or intend to work as a software developer, you should ask yourself why — most likely you're either you're missing something you really should know, or you're lucky enough to be working within a narrow area where your deficit doesn't matter."
Talk about reading just part of the sentence.
yes you did...
That being said, I honestly prefer being judged by someone as accomplished as Colin Percival than hiding behind the attitude that we should not judge other people's abilities.
Let's face it, there's an abundance of downright bad and mediocre programmers. There aren't that many top-notch programmers, although each of us would love to think we're among that small group. This is what motivates us to speak out against any attempt to measure and assess abilities, especially if such an attempt is likely to rank us low.
I'm not ashamed to admit that, so far, I haven't been able to formulate an answer to more than 5 questions out of 10 he posted so far. While I freely admit that I don't consider myself to be a programmer in Colin's league, I think that's really beside the point here.
My knowledge is cached: what I work with all the time is easier to recall; what I've worked with or read about, but I'm not using all the time is not as easy; and, of course, there's a ton of things I don't know, have never worked with or read about. This means that I don't feel confident talking about B-trees without consulting Internet, but since I read about them previously, I do remember that they're a form of search trees and that databases use them (or their variations) a lot.
What's the point of this whole rant? Let's bottom-line it. I'm probably not as good as Colin overall and I'm definitely not as good as he is at what he, specifically, does. That doesn't mean I should feel threatened by his exam. Nor does it mean it has no validity whatsoever. Nor does it mean that Colin is "difficult to work with" or fits any specific personality type.
Those two claims about Colin's personality and intentions are what really bothered me about your reaction. I'm not a fan of straightforward ad-hominems, but I detest veiled ones even more.
Having said that if these questions tickle your intellectual curiosity go ahead and figure out their answers, just throw away the belief that you can become a valid/invalid software developer just because you can/cannot answer some questions.
Is this what we are coming to? The dumbing down of software development? Have the mainstream anti-intellectualism been spilled over into software development?
I didn't say "people who know the answers to these questions are difficult to work with", I said, "people who declare their knowledge as an absolute requirement are difficult to work with".
(So what's a good question to see if somebody understands cache coherency? Maybe describing a problem caused by cache-line contention, and asking for an explanation and fix to it?)
On the other hand, I know nothing about pthreads, made a handwavy guess about that question, and got it very wrong. The underlying failure was a bad guess about the semantics of condition variables. I could whinge about that, but I bet that if I spent more of my life writing multithreaded software -- which is, make no mistake about it, an important skill -- then I wouldn't have made that wrong guess even if I'd still never used pthreads. My answer might still have begun "I've never used pthreads, but ..." but that would have been followed by a better guess than it actually was.
Something may look like a trivia question but give much more information than just "does this person have the information stored in their brain right now?". In this particular case, Colin could (if he chose; he probably has better things to do than go into such detail on the hundreds of responses he's getting) guess that I haven't written much multithreaded software, don't spend a lot of time optimizing things for the memory subsystem but have a good grasp of principles, and am good with algorithms. Not so bad for three trivia questions.
And, whatever cperciva's failings (which may for all I know be many and serious) one thing he certainly isn't is a "B-player". (But then, in my experience talking about "B-players" is itself a bad sign.)
I don't think the questions are awful, though they do tend to have a trivia component to them.
What I think has really happened is that the whole thing is completely mispackaged. By calling it a "software development final exam" and saying it's things every programmer should know, he's set up an idea that it would be fairly broad and comprehensive, when in reality it's fairly narrow.
Plus that terminology is a bit socially off-key since it sets people up to be defensive rather engaging in meaningful discussion.
Maybe B-player is harsh but in my career I've worked with some very good programmers and some real stars. When I talk to the very good programmers I always feel dumber, but whenever I talk to the real stars I always feel smarter myself (though heaven knows that's not actually true). This seems to fall firmly in the "very good programmer" category.
Need a car analogy? Fine: you pitched this as "what every driver should know about driving", and you're talking about the intricacies of combustion engines. That is to say, it's interesting, may pin together some pieces of other knowledge, but of no real practical use to anyone who doesn't also fit in to the "and I'm a mechanic" demographic.
2 of the questions are about drift racing (mutexs) 2 of the questions are about the design of the fuel injection control chip 1 question is a subtle question about the fuel injection control chip
Programmers affect and determine the operation of the computers their code runs on much more than a driver driving inside the lines.
i.e. the developer who chooses the slower zeroarray algorithm that still completes in reasonable time, but provides a better user interface and better feature set is still the better software developer (but perhaps the poorer computer scientist).
I got just over 100 responses to part 1, and I'm almost halfway through grading them. I'll be sending out a batch of emails (form emails, I'm afraid, to save time) once I've caught up with the part 1 grading.
I will be posting my answers, my grading scheme, and an analysis of the results next week.
I will not disclose any individual's performance (except to them) without their permission.
Thank you for that. :-)
I have a BS and a master's degree in CS and I didn't know about MESI cache coherence. And I've never run into it in 7+ years of professional experience. But that doesn't mean it's a useless question. I don't feel compelled to learn all about it right now, but at least I'll get acquainted with the Wikipedia-level facts. One day I might study it further.
Is it a useful question, just because it was asked?
I'll admit that I couldn't answer many of these questions off hand; I hope the criticism doesn't come across as bitter.
Contrived example: Someone who came up with a quicksort algorithm independently, and is completely aware of its operating conditions, but chose to call it mysort instead, would be unable to answer the question in the last test despite having all the theoretical knowledge needed.
There is, of course, still something to be said about being able to use terminology that is shared with others, but that changes the nature of the test in ways I'm not sure the author intended.
OS questions would involve process/thread/fiber, scheduling, priority, multitasking, memory management, virtual memory, IPC, deadlock, distributed deadlock, interrupt/signal, input system, output system, GUI system, file system, disk system, IO caching, driver model, kernel space and user space, privilege and non-privilege operation, security model, user/process permission and privilege, file permission, authentication, authorization, etc.
I covered locking, race conditions, virtual memory, and processes; I think those are much more relevant to most developers than knowing the internals of how filesystems work or how interrupts are routed.
Some of these questions aren't even necessarily relevant in getting a computer science degree let alone software development.
The point is that knowing the answers or being able to figure them out on your own is, I believe, strongly correlated with having a good understanding of the entire area the question is drawn from.
For example, suppose you let someone google around for a half hour to try to answer the question about how to determine if a graph is bipartite. Now, you spend about a half an hour doing an oral exam to see how well they understand what they just regurgitated.
I suspect that you would see a wide range of performance, but that people with certain academic backgrounds might do much better than others. That's the "more interesting question" that I had in mind.
Actually, suppose someone had never take graph theory came up with a novel but ultimately flawed attempt at an algorithm. That might be a stronger sign of talent in this area than someone who had taken the class and was able to reproduce an algorithm (even if that student showed a genuine understanding of it).
For instance, suppose someone doesn't really remember the quicksort algorithm, but looks it up and is quickly able to determine the run time by analyzing the algorithm. To me, that's pretty much as good as knowing the algorithm's run time off hand. Maybe even better. For all I know, if you changed the question just slightly and ask if the run time has changed, the first student has shown the ability to analyze the run time of an algorithm - the second student's ability to do this is still unproven.
These aren't necessarily the questions I'd use in an interview -- I don't have the luxury of reading someone's answer and then probing further.
zeroarray2 jumps around (0, 1024, 2048, 4096, ..., 1, 1025, 2049, 4097, ...) in a way that intentionally makes life difficult for the cpu's cache manager, and generating lots of cache misses.
edit: Ignore. I'm wrong, see below
I could understand a difference for two-dimensional (or more) arrays, where different languages lay-out the array contents in memory differently. Does Fortran lay-out the contents of one dimensional arrays in an unusual way?
for(i=0...
for(j=0...
A[i][j]=0; for(i=0...
for(j=0...
A[j][i]=0;The core of the issue is with cache-friendly access patterns. This is a simplified explanation, but here goes: CPUs only have so much cache, and the processor needs to keep enough data in that cache that it won't be left waiting for too long before the next batch of data requested from ram is available. To that end, when you access a chunk of memory, the CPU will grab the requested region and then some, hoping that most of your work for the next few microseconds will be within that region. The second implementation jumps by 1024 x sizeof(double) at each access (so, on my system that's 8k), which is plenty far to blow through whatever memory was prefetched with your access, and in so doing force the processor to sit around and wait for another memory location to be cached.
There's another, probably less significant way in which this is a pathological access pattern: alignment. When, in the first implementation, the processor operates on contiguous blocks of memory it is free to prefetch memory in nicely sized chunks that begin and end on convenient numbers, which is important because memory today is nothing if not a tower of multiplexed access -- asking for a few extra bits over the edge of a row, wherever those borders happen to be for your system (probably the width of the memory bus is a good guess), may not seem like much, but if it means the memory controller has to access a row of ram that it otherwise wouldn't, that involves first writing the data in the starting row, then waiting for the appropriate delay to save that data before switching rows and repeating the process. Looking at zeroarray2 in that context, you're asking the memory controller to save sizeof(double)*8 bits of zeroes, a fraction of a row, at once before moving on to another row only to revisit the first some time later.
Answers to the questions pertaining to memory (rather than concurrency) can all be found in Ulrich Drepper's excellent tour of contemporary memory systems, "What Every Programmer Should Know About Memory" ( http://www.akkadia.org/drepper/cpumemory.pdf ).
edit: fixed accidental italics, added useful additional reading link.
I would guess that the example was explicitly contrived so that prefetching would be useless.
Yes -- even if prefetching could happen, the first function would be faster than the second, but I thought removing the whole issue of prefetching would simplify the question.