ericlippert

Well, my intention was not to sound pompous, but writing is hard, you know? Thanks for your cogent critique of my tone.

Well as I said in the article, the intention of the question is emphatically not a "gotcha" about how they are actually doing. It's intended to find out if there are any problems that need addressing.

These are good points.

The number of candidates who have told me that they want a computer rather than a whiteboard is small but not zero.

I do give the same few problems over and over again, and I do take extensive notes, but I am not keeping track of a specific set of metrics that I track across candidates. I have a pretty good sense of where the "middle of the pack" candidate is. The process could be more scientific, yes.

Thanks for writing that up; really interesting stuff. I agree with many of the critiques you make. I suspect that it may be easier to get the sort of standard process you describe implemented in a relatively small organization that has a relatively consistent set of products and commensurate challenges than in a large, diverse organization like Microsoft, where I learned to interview. The Microsoft interviewer has the challenge of finding candidates that are good both for the specific team they're interviewing for, and also have the general skills necessary to work with other teams in the future. As you say, it's a hard problem.

That's interesting feedback, thanks. I have found that most of the time people are happy to talk about their work, but I have also encountered the flustered reaction you describe.

I usually take a photo of the candidate's work, but not to evaluate whether it is correct or not; I already know that. I take the photo primarily because the board is about to be erased for the next candidate, and I want to make sure that my notes are accurate when I describe the candidate's strengths and weaknesses.

Having a photo also allows me to compare the candidate's work against that of previous candidates, so that I can ensure that my interview process is well-calibrated.

It is also a preventative measure, so that when a no-hire candidate sues the company claiming discrimination in hiring practices against whatever protected class they are in, I have more evidence than my notes and fallible memory. Fortunately I have never actually needed to use that.

I agree; a problem I often give -- which I will discuss on my blog next week -- is "here's an existing solution to a problem, it is full of security holes and bad programming practices, describe to me some techniques you would use to fix this code without changing the public interface". That's problem people face in real-world development all the time: bunch of code, works when callers are benign, badly broken when callers are hostile, the client has shipped so we can't change the interface, fix the problem.

Oh yes really.

I have had candidates with PhDs in computer science who had a deep understanding of static analysis of programming languages, but who did not know how many bytes were in a pointer on a 64 bit operating system, who thought that there was an algorithm for compressing any 64 bit number into a 32 bit number, who could not describe the possible consequences of a buffer overrun in C, and who had no idea whatsoever what a virtual function table was.

Now, to be fair, as Dijkstra is said to have pointed out, astronomers are not experts on telescope building. People with deep knowledge in one small area of a field are often ignorant of other areas, even closely related areas. In those cases it's even more important to get a sense for how the candidate approaches problem solving, whether they can learn on the job, and so on, because you know that they are going to be deeply in problem solving mode very quickly when faced with their first real-world problems.

I hold up myself as a case in point; I helped build the JavaScript engine that was in Internet Explorer in the 1990s, but know practically nothing about HTML, the browser DOM, CSS, and so on. If I were interviewing someone like me for a web developer position, I would have to be convinced that the candidate could learn quickly.

Asking people to solve riddles is of course silly, and everyone I know at Microsoft stopped doing that fifteen years ago.

I have had several candidates with established work history who were turned down on the basis of their coding work during technical questions in interviews. For example, I once interviewed a candidate who had written a popular developer tool for the Mac; the candidate was applying for a job on a developer tools team and had an excellent resume. And did not know what a binary tree was, did not know what recursion was. Though success in industry may be correlated with ability to solve the most basic problems one faces every day when building the semantic analyzer for a compiler, it is by no means a perfect correlation.

Demonstrated ability to take the specification of a simple, recursively-defined property and translate it into two lines of correct code on a whiteboard is a pretty good correlate.

And of course, as I pointed out in the article, I am much more interested in how the candidate reasons about the code, not whether they can write two or three lines of correct code on a board, or type them into a computer.

I have tried that as well in the eleven years since I wrote this article. It is much more work for me to set up a development environment, get the monitoring software working, blah blah blah, and there are many points of failure, but if the candidate feels more comfortable doing it that way I'm willing to do so.

The problems that I ask are so easy that it really does not make a difference whether the candidate is writing on a whiteboard or typing on a computer; most of the problems I ask can be solved in fewer than six lines of code. I am using the code as a starting point for the conversation, not the end point.

I take it into account by asking a problem that is so simple and straightforward that you do not need google, intellisense, or any other tool to solve the problem. As I mentioned in another comment above, I have asked problems as simple as "add and subtract four numbers". A candidate who needs to use Google to determine how to add and subtract doubles in C++ will not be successful.

Now, perhaps I am alienating candidates, but first, I am very willing to accommodate any candidate who wishes to demonstrate their coding skills to me in some other way, and second, my primary goal is to prevent bad hires. If good hires go to the competition, that's too bad, but it is way more important to prevent a bad hire. What I am not willing to do is fail to test candidates on their coding ability; I have had candidates -- some with advanced degrees -- show up who had no demonstrable ability to solve practical problems.

The problems I ask are not amenable to solution by searching Google, and frankly, I am not actually interested in the solution. The problems I give are so easy that someone who cannot solve them by writing the code from top to bottom in one go is unlikely to be successful. The problem is a starting point for a larger conversation about modern software development.

For example, I used to give a question where the solution was literally: take two pairs of numbers, find their differences, add the differences together. (The problem involves computing the difference between two times represented in an unusual format.) A candidate who is unable to take the differences of two pairs of numbers and add them together is going to be unsuccessful -- and I have had candidates show up in my office claiming to be "9 out of 10" C++ programmers who could not write correct code that added and subtracted four numbers. The vast majority of programmers will solve the problem correctly, and that is step one; I want to know how are you going to test it, can you prove that it is correct, if the times were in different time zones how would you change the signature of the method, should the method return an error for invalid inputs... I want to know if the candidate can actually reason about code.

Something I did not mention in this article -- which I note I wrote eleven years ago -- is that in the time since I have occasionally had candidates say "I get anxious when writing on a whiteboard, can we do technical questions on my computer?" In those cases I have gone to the extra trouble to set up a problem on a computer, and then had the candidate explain their solution to me.

So I have tried it both ways, and I see very little difference in the outcome. For example, I give a problem on detecting whether a tree is out of balance. A great many candidate makes the same bug; they check to see if the subtrees are balanced without also checking to see if the subtrees are of similar height. They make this mistake whether they are writing the code on the whiteboard or on a computer, because the error has nothing to do with how you are writing the code, it has to do with the thought process that happens when trying to translate a clearly-written specification into one line of bug-free code. Candidates who are informed that they have a bug solve (or don't) the same way on a whiteboard or a computer: by reviewing their code, re-reading the specification and thinking about test cases.

Indeed, though the iron crucible I use gets hot enough to be red hot. Whether the molten aluminum has a faint red glow about it or not I cannot say; I don't usually melt it at night.

Many parts of machine tools like lathes need to be as flat as possible, because if the surface of the tool is not flat then that error is transferred to the object you are tooling. To get, say, a lathe bed flat, you buy or make a reference flat surface, paint the lathe bed blue, and then rub the surfaces together. The parts where the blue rubs off are the high parts. You take a scraper -- basically a modified chisel that takes off a thin layer of metal -- and scrape down the high parts. Wipe off the paint, repeat until there are no more high spots. It is very tedious.

As I mentioned in another comment, ironworking requires taking things to the next level in a pretty serious way if you want to cast it. Iron melts at 2800 degrees, you need ceramic crucibles, it is heavy, and so on. I'm not going to even consider iron until I have a lot more experience working with aluminum.

That said, I do have some friends who are experts at working iron, so if I do decide to take it up, I'll have a lot of good advisors.

I'm going to try to get good at working aluminum before I move on to iron; that's a whole other level. Fortunately I have some friends who are master ironworkers so I will go to them for advice should I eventually get there!

Exactly. It is extremely inexpensive -- the majority of the aluminum I melt I found abandoned on the side of the road -- it melts at only 1200 degrees F which is an easy charcoal fire. It is easy to machine, doesn't corrode, is hard enough to build interesting things, and lightweight.

Correct; the oxide is not a problem. There's not much of it, it melts at a much higher temperature than the pure metal, and it floats, so you just scoop it off before you pour.

Whether oxidation or reduction is taking place during the melting process is determined by the amount of oxygen arriving via the air blast; too much oxygen and you get a hot oxidizing atmosphere, which oxidizes both my iron crucible and the top of the melt. Too little oxygen and you can actually get a reducing atmosphere, the furnace runs a bit colder, less heat escapes through the flue. So it's a tricky balance to strike and I'm still fine-tuning it.

I add potassium chloride and sodium carbonate to flux the melt and remove dissolved hydrogen; the KCl helps keep more aluminum oxide from forming apparently, though some home foundry sites report that they don't see a difference if they skip that step. Both chemicals are cheap, so I'm not too concerned.

Thanks for the kind words, I appreciate it. Indeed, I did a lot of research on home foundry setups and tool building before I started this project, and I am deliberately going slowly and carefully.

Indeed, I watched all those youtube channels for quite a long time before deciding to build a foundry; there is excellent stuff out there! The Gingery books are really very good for books, but for a lot of these tasks it is so much more clear to watch someone do the work.

On a more personal note, I'll add that I find it ironic that you accuse me of meanness, contempt and belittling others, because I find that rather mean, contemptuous and belittling.

In the future if you have a problem with something that someone has written on the Internet, why not contact them and have a civil conversation about it? It's not like I'm hard to get ahold of.

If in the future you do that, you might find that you've misunderstood the motivation of the article you find so objectionable. And the author might find out from you that they could communicate their point in a better way. The world would get better all around, and that could all happen without resorting to profanity and name-calling in a public forum.

I do not hold students in contempt. My article is an honest attempt to help students, because, as I note in the first few paragraphs that you object to, their current course of action is one that is not helping themselves.

When I was a CS student at Waterloo, no one taught us how to debug small programs. Maybe things have changed since then, but at the time this was just a skill you were expected to pick up for yourself. I remember many times coming into the lab early in the morning and finding my fellow students asleep on their keyboards, having stayed up all night trying to find a bug in a 20 line program.

If you don't like my article then that's too bad; my suggestion to you is rather than complaining about it, why not write your own that you like better? I'd be happy to publish a link to it.

I am neither high-profile nor an executive.

There were many factors that led to this decision. I described the primary factor in my posting: I've been at this job for sixteen years at the same company, I am turning 40, and it is very common for people at that age to want to make a change.

I assure you that "how C# is treated" was in no way a factor; C# is treated extremely well at Microsoft. Moreover, think it through: I would not have taken a job that continues to improve the broader C# ecosystem if I thought that the .NET platform was some kind of dead end. It is vital.

I'm glad you enjoyed my answers; however, I am never intending to come across as smug. It is difficult to get across subtleties in an impoverished text-only medium; it has certainly been my experience that instructive and constructive criticism can easily be read as smugness. It's also been my experience that trying to deliberately make it sound less smug often just turns it into sounding like condescention. Text-based communication of technical issues is a hard problem and I do struggle with finding the right tone. Thanks for the feedback.

Of course Feynman knows the intended answer. He's way ahead of the interviewer, and is having some fun with him by deliberately avoiding the "correct" answer.

For the record: I intended no disrespect; I have immense respect and admiration for Feynman.

And for the record: I do not think that I am as smart as Feynman; that would be ridiculous.

The piece was intended as satire, and is in a long tradition of such dialogues intended to ridicule a "straw man" of a particular position. Consider, for example, Gallileo's Dialogue Concerning Two World Systems, in which the scientist, Salviati, criticizes the position of Simplicio, who believes the earth to be the center of the universe.

I couldn't think of a better modern figure to stand for science, reason, clear thinking, and a mischievious sense of fun than Richard Feynman as my Salviati, and I hope that he would appreciate the spirit in which it was presented.