99 Out Of 100 Programmers Can’t Program – I Call BS
skorks.com
skorks.com
Clap clap clap.
The same people that insist that the minimum development station is a tricked out computer with two (three?) big monitors, SSDs, the best IDE money can buy, etc. don't understand why someone is uncomfortable writing a program on a whiteboard or, even worse, over the phone.
Unfortunately, I don't have the solution. Perhaps pg can add "Interviewing that doesn't suck" to his list http://ycombinator.com/ideas.html - obviously this is included in "7. Something your company needs that doesn't exist." and is the company version of "8. Dating."
Personally, I think Steve Newcomb has a 90% (maybe better) solution. http://blognewcomb.squarespace.com/essays/2010/10/14/cult-cr... (discussed here http://news.ycombinator.com/item?id=1793095).
We also ask them a design question, typically something like "design an automatic retrieval system for one of our robots". Again, not being mechanical engineers we have no way of telling whether their ideas would actually work. It's the thought process we're interested in looking at: how detailed they can get, how they anticipate and solve or work around problems with their design, etc.
Given that we have a pretty top-notch software team, I'd say our system has worked out rather well. :)
Most companies employing programmers today are not such companies -- they cannot possibly hope to hire A+ employees, but they are trying to build a solid team of B- or C-level programmers, just hoping to keep the F-level out. As far as I know, this is a total crapshoot and the ones succeeding in it do so mostly by luck.
On the other hand, the same companies that are decrying "99 out of 100 cannot program" also claim to be in the same high-end tech company space as Newcomb.
I had a set of coding questions to give final round candidates to complete in person. By paper, whiteboard, or laptop -- their pick.
The success rate for candidates was about 25%.
I then started giving the same questions as a time-limited take-home test, for first-round candidates.
The success rate jumped to almost 100%.
Same questions. Less-filtered candidates. The difference? The primary determinant of success was interview environment, not candidate ability.
It's becoming more of a problem, especially with university level recruitment, that candidates are either getting others to take the test for them or downloading solutions from somewhere.
Often candidates once bought in for interview fail to be able to explain how code "they wrote" works.
Some company wanted me to write a simple string reversion and prime number finder programs on paper, I failed because I'm not used to code on paper.
Some other company wanted me to write a server client program which takes a file from client and saves it to server. I completed it on my laptop on the same day when I got home.
If the interviewer is going to make the interviewee take a written test; it must be about concepts, designs and stuff, not implementation.
The "naive" solution typically is a sieve method and checks more than the required number of factors (goes beyond sqrt(n)). It's like asking someone to sort an array and expecting bubble sort because you never use sorting algorithms in what you actually code.
The best way I know (and there are actually better) to find primes is using an approximation algorithm that no one would ever come up with independently without months of research. The Miller-Rabin primality test finds if a candidate for primality n has a % chance of being prime by looking for numbers a < n that don't satisfy Fermat's little equation. Almost every composite number violates Fermat's little equation (Carmichael numbers are the only exceptions).
Once you hit the desired probability that n is prime (testing more a < n raises this probability) you use a deterministic method to determine 100% if n is prime. If it isn't you throw it out and use Miller-Rabin to find another candidate. Even though you might throw some candidates out and have to pick others to check for primality this method is far faster than a purely deterministic solution using a naive sieve methods.
This is a moronic test of ability. It exposes the arrogance of the company in assuming they can evaluate candidates in 15 minutes by asking the most challenging number theory problems out there when they have no understanding of the current methods in the discipline.
It is exactly like asking an applicant to sort an array and expecting the naive bubble sort response to check they can develop and implement a naive algorithm for searching. It says you can brute force a solution with a for loop. It says nothing about how well you understand advanced data structures, recursion, algorithmic analysis, language features, domain specific knowledge, program design and structure, or knowing when you you're out of your league and should google the best approach. It quizzes chapter 3 of any intro programming book and basic division in a situation where you would never use it.
Ask them about event handling or something useful like hash tables or linked lists or version control systems. Any math major can answer this question easily without knowing much beyond for loops and arrays. It tells you almost nothing about their programming skills. I was math/cs with grad school in math and every math major I know could do this. Most of them have no idea how to code. I know some great developers who would struggle and fail because they typically don't use this math in their job and have forgotten it. They work at some of the top companies out there (which are frequently discussed on this webpage).
I can't believe in 2010 this isn't "conventional wisdom", but heck, take advantage of the competitive advantage while you can. No matter how impressive your linked list reversal is on the whiteboard, running code is even more impressive.
Next, I ask you to parallelise it, and you don't recall the pragma that OpenMP uses, so you just need to pull up the manpage, or you don't have the Actors library installed, so, uh, what's the password for wireless? Oh, and you still don't have that large file, so even if you get your parallel code running before I run out of patience, waiting for the answer "well, access to this counter would have to be synchronised, so that would create a bottleneck", running your program will still not have enough data to munch on to gain any insight in it's performance.
tl;dr: You will have to be very sure of yourself to pull this off. I want to know how you think, not how you handle your tools.
You're seriously arguing against having somebody actually code in their interview? I'm not saying it has to be the entire interview, but how incredibly crazy is it to actively argue against the idea that maybe you, as the interviewee, might want to actually show you can do what you say you can do?
Besides, I find myself questioning if you've been on the interviewer side of very many interviews. Finding people who can slurp a file in at all is a damn good start, unfortunately.
Then again, I know I do this really whacky thing where my interview isn't set in stone, but I actually react to the skills demonstrated on the other end of the table. Believe me, if somebody walks in sporting a laptop I'll customize something up for them to show me out of my base set of questions. It would hardly be any different than what I do anyhow.
And I can do without the ad hominem as well. No, I haven't done a lot of interviews (about 6-7 I think), but my post still has a substance that can be argued against.
> You're seriously arguing against having somebody actually code in their interview?
No, I have no problem with coding in an interview (I think it should be part of it, just not the part where you do something else). I have a problem with generalizing the advice "When asked to do whiteboard programming, whip out your laptop.".
Personally, I hate to program on someone else's computer. I run pretty bog-standard linux with minimal plugins and aliases (in part, so that I'm comfortable in a bog-standard environment typical of servers). The problem is, lots of people have customized the crap out of their system. The result is added discomfort when I try to do something on their computer.
Don't assume everyone is like you.
This is even more important in a small company. A small company cannot afford to hire a dead weight. A couple of those will kill the company -- they waste resources, don't get work done, and, what is perhaps very dangerous, drive away top technical talent who does not tolerate mediocrity.
The interview process should be extensive, yes, but it should also be spread out in a way that allows you to fail candidates early. Blocking out a day of interview time at once can be inefficient - rarely will that person be sent home early (for various reasons) and they are now just wasting a bunch of peoples time, and their own.
You presumably want people who will enhance and help your top talent.
But with a small company, when you have 5 people and you hire the 6 and it turn out to be a bad choice, it could kill the company.
Hiring many people at one time I think is a sign of a problem. We try not to hire more than 3-5 people / year. It is not that bad for a each top dev to spend 10 hours / year to find the right candidate.
Before the day long interview we would have a phone interview. At that point you would decide whether you want to take a day out of your life to find out more about a company where will potentially spend years working. I don't think that is that unreasonable?
The point is, if you expect me to invest so much in an "interview" with you, you had better be laying some big chips on the table as well.
Remember you can always leave and we can always stop the interview.
Well this way of interviewing has worked well for 15+ years ;-) , we like it, we'll probably stick to it.
After being there for 4 years, I would it was the best 1 day of PTO ever spent in my life.
Exactly. I loathe whiteboard interview questions. In five years of programming, how many times have I ever written or looked at code on a whiteboard to solve a real, work problem?
Zero.
Yet some people think the best way to learn about a candidate is to put them in a situation far removed from a real, daily work situation and see how they perform.
If you don't use whiteboards in these ways, then I'm going to suggest putting whiteboards near developers and consciously try it. You'll be amazed how well it works.
Really, if you can't write a few lines "on the spot", what use are you going to be to an employer? In the real world there is always some deadline. I don't always expect perfection in a whiteboard scenario, but there's a world of difference between a blank stare and a 90% correct solution with an off-by-one.
But in the end, I like this kind of interview, because I don't want to work for anyone who would evaluate me like that.
I'm hoping to finish my PhD and become a professor in Computer Science. My plan, given university constraints, is to never once offer a written final. I've not once taken an exam where I was confident just writing code down. It doesn't make sense. Offer them a final on a computer with Internet access, but with a sufficiently difficult problem that it can't just be Googled for a complete answer. Their web access logs are turned in with their exam, so I can see if they copied something if it looks suspect, or if they're borderline I can give extra credit for going through a mental process that shows promise.
That is what makes the whiteboard such a good tool: when working on a whiteboard, you can focus on the problem, not on pesky details. A whiteboard will never complain about API issues, function naming, etc. Not even by using squiggly underlines.
In school we were taught to work through algorithms and other design using pseudocode and pencil and paper. Doing this also helped reenforce that languages are very interchangeable using the same algorithms and designs. I wonder if that is simply not done anymore in school?
Another option is to tackle the interview itself as a challenge and practice interview skills, such as whiteboard coding. I think Steve Yegge suggested this in one of this blog posts about how to interview at Google. Personally, I find this to be very impressive: You knew there was going to be whiteboard coding, so instead of complaining about it, you practiced it. If you (casually) mentioned this during one of my interviews, I would probably give you some extra points.
Also stressful deadlines are an indicator that sosmthing else is wrong in a company. The company should fix that issue instead of hiring people that can "program under pressure".
1) Bug in Google authentication which allows websites running on App Engine to grab users email addresses without permission
2) Crunchbase making all historical searches public
3) Facebook downtime bringing down "like" buttons
All three of these issues were for the companies in questions bugs that had to be fixed in realtime. Does it indicate that something is wrong with these companies ? - no.
Any changing production system is going to suffer from bugs that on occasion will require programmers to be woken up in the middle of the night to urgently fix a bug.
It means that their testing wasn't very good.
If programmers are being woken up to fix something then something is very wrong with an environment.
Look at cryptographic libraries. Often they've been gone over line-by-line by hundreds of experts in the field, have extensive testing both unit and field. Yet bugs still get found in them.
Spotting bugs in code, operational questions, and writing code samples are probably the closest you'll be able to come within the time restrictions of an interview.
Needless to say, actual software development in the office is pretty relaxed. But when you go out on ops, you will be in the pressure cooker and you will need to handle it.
The reason that big monitors are useful isn't because it makes you smarter or more able to reason about trivial problems. It's because it provides more space to cross reference code, more space to look up documentation, and so on.
For the sorts of questions that are being asked in an interview, you should be able to sketch out a solution on a whiteboard no problem.
Most programmers in my area are no longer care about Algorithm, Data Structure, Operating System principles, Database principles, or basic Math.
Most companies, that I work for or heard from employees, don't have good codebase, don't have good automation tests, don't have good culture.
Sometime I do wonder what does it mean to "be able to program".
Can deciding to use MVC (Spring, ASP.NET, Rails) to spit out XML/JSON so that it can be consumed by AIR, pure JS/jQuery, or even your iPhone app be counted as "be able to program"?
Can writing good code that can be easily testable be counted as "be able to program"?
Can producing code that consumed WebServices from other payment provider be considered as "be able to program"?
Or, can solving tricky algorithm problems is the de-facto "can program" filter?
Some of the "can..." produce business values. Some others, might not necessarily produce business values, but instead, just to show that "yeah, I can sling code the way you want me to".
Tricky field...
Please don't enumerate buzzwords like that. Or at least ROT13 it to be kind to some people's refined slacker sensibilities.
Right now I am digging my nails into the desk and biting my teeth: please, don't give the scumbags any of your attention. Hack outside what you see on corporate job descriptions, even if you have to wait tables to support your console habit.
Everyday you spend learning something to meet a job requirement, you're being someone's #&@! And someone who doesn't know you or give a crap about you. Do this shit for yourself. Do what you* want.
Forty years from now, at the end of your career, what do you want to remember? What do you want to be remembered by? Surely, not that pile of inventory reporting macros you wrote in Vector SQL-PowerG# 2020.
I see these ghosts at bookstores, discussing how they're certified in Java, Oracle, CISCO, and A+, and now need to add Ruby and Scala because they saw it on an ad. I am sure a great many of them can code and code "well", as much as their tools allow them. But at what cost to themselves, their ego, karma, mental health, personal growth, and fucking COJONES .. tienes algo?
Don't choose to remain adequate, and acceptable. Aim to excel, exceed, and if necessary, offend. Today's challenges and good times are tomorrow's fond-memories. Take a shot at something great, even if you fail, do it, and enjoy yourself.
But a couple days ago I told myself to ignore the rat race. I then bought a Linux book specializing in command-line, a Vim book, and a Rails 3 book. (Why Rails? um, why not? it's better than those corporate tools). I don't care if I have to learn PHP. As ugly as it is, it's still better than using .NET tools (regardless if it's ASP.NET MVC or not).
C# could be a better language, but I just want to use light-weight tools (even if it means I have to learn many-many tools). No more heavy weight tools.
I noticed that the people who choose the lightweight path (C/C++, or LAMP/RoR) seems to be able to focus on what matters the most: learning skills that can last longer than .NET version 2.0.
Instead, he made sure never to actually be useful to the programming or hardware teams, but talk to them just enough to know the basics and keep updated. Since he wasn't actually ever working on anything, he used his free time to mingle with management and used his inside knowledge from the programming teams to announce potential problems to management before the actual programming teams could communicate this info.
Sure enough, he slowly moved into management and higher salary brackets by never actually learning how to program. Of course, the programmers and hardware teams caught on to his game and cut him off from any useful information, and without any actually knowledge he hit a ceiling within the company pretty quickly. Still, it was an important business lesson for me, and ever since watching this all take place I've made sure to carve out some time from my heads-down work to maintain visibility and directly communicate with management.
College only teaches you theory, for the most part. Some will not teach you any programming besides some generic C++. You have to learn that on your own; I'll believe that the vast majority don't have the drive to learn that on their own for years to perfect their skills.
I really agree with your points on the problem being with the interview process though.
The fact is that computer science is not software development, any more than you could take an astronomer, drop him on a ship and say "navigate!".
It doesn't matter what the original program does. I cannot count how many times I started with a program that printed Hello, world and wound up with with something useful and unrelated.
Personally I write all my makefiles from scratch. I'm always baffled by other makefiles though. They always look so huge and for some unknown reason just overly complex.
Contrariwise, if he has to write something from scratch he can... as long as it's entirely stand alone and not too complex.
Let's face it: our industry is as bad as the quality it produced.
And this is basically for the equivalent of an undergraduate's degree. I'm not saying all degrees should have this level of variety, but are people really coming out of Uni without at least having written a linked list in a low-level language?
Not once did I use an MVC beyond a crappy Java implementation. We didn't do maintenance practice, or log file spelunking, or customer interaction.
I learned how to create multi-user, networked, stateless applications editing one source. I created a compiler and language, learned assembly, studied algorythms and z-schemas. I switched between languages every day and solved some annoyingly abstract and difficult problems.
Number of bugs I solved in other people's code outside of an exam? 0. Number of incredibly obscure projects I took from start to finish? 0.
CS Degrees only teach people to be potentially good programmers IF those people are able to learn how to fix shit by themselves. Otherwise you're teaching greek to a deaf person. Sure, they know what they're asked, but they can't talk back.
Agreed. My experience was that I got into programming as a kid. By the time I enrolled in college, I had written hundreds of thousands of lines of BASIC code and had written a ton of Pascal programs. College enhanced my programming abilities with concepts like big O notation but for the most part did not give me any hard technical skills that were hirable. My first job out of college was programming in Java which I picked up on my own and via an internship.
I see two likely posibilities here. One, you classmates were all terrible -- i doubt that is true, but maybe one or two never 'got it' in industry. Two, most of them discovered that they didn't love programming, and if you don't love this line of work, it's really easy to hate it.
Maybe they just found their passion elsewhere?
I was thinking about it recently, if I had spent my 4 years of uni (just finish up now) focusing solely on the course content and not spending time reading things like books and hacker news and hacking my own projects/ freelancing/ startup, where would I be as a programmer? I think I would be so far behind where I am now, especially on the business side of things but also the programming. I know for a fact the thesis I'm finishing up would be nowhere near the level I have go it to.
It's interesting that in uni in terms of ability I would be close to the top, but when I've seen some of the amazing talent in Melbourne through coworking I'm just average.
I would rather say, "Grades" are not that important. You don't go to Engineering school just to learn programming.
Programming is something you learn to apply what you are learning at school.
Maybe it's the dogmatic zealotry, but it seems like 9 out of 10 programmers just repeat what they hear to be common wisdom so that they don't stick out like a sore thumb. Who would want to be one of these mythical programmers that cannot program? Who wants to be called "dead weight" and cast out of the group? Who would ever admit to being one of these fabled creatures?
We programmers need to stop being so nasty and elitist. Programming is hard and I'd be impressed if you could write down a working solution to a toy computer science problem you probably slaved over for months in university on a napkin in 10 minutes. You should go on Jeopardy. You'd make a killing. I can't do that; I don't have a mind for trivia. We need to stop entertaining our egos and thinking of everyone as "dead weight." We can all stand to improve the code we write. We should be encouraging one another and learning together.
We don't "interview" people anymore. We "screen" them. I suspect it has something to do with the elitism in the industry as well as the inherent anxiousness and dread of interviewing people you don't know. It's become a terribly broken process, yet it seems precious few people such as the OP realize this.
We could do something about this. Here are some tips I have from my experiences:
1. Know the person's name before they walk into the room. Don't shake their hand as you look down at their resume to figure out their name. At least make it look like you put thought and care into who you invite into your office for an interview.
2. Don't use a white board or a pad of paper. A select few eccentric programmers have ever actually sat down and wrote out a program on paper. The tools of the trade are debuggers, compilers/interpreters, and text editors. If you want to see how they approach problems, tell them to come prepared with code samples to share or something. Make sure they're aware that they'll be talking about their solution, the tools they used, etc.
One last thing (and a bit of a shameless plug), I think software can be used to ease some of the pain of the hiring process. I'm working on a project to amalgamate meta-data from repository analysis tools and social networks into a single candidate profile that should give you a pretty clear indication of their skills and abilities. The idea is that you should only invite people you are interested in hiring into the interview. To be able to do that you need to know as much about this person as you can and be comfortable that you have an accurate picture of their abilities and work history. The volume of resumes can make this hard... why not automate it?
Just sayin'
Every physicist can solve first-year physics problems on the back of a napkin.
Every mathematician can solve first-year math problems on the back of a napkin.
Every machinist can sketch her approach to making a widget on the back of a napkin. Electrical engineers can sketch schematics. Chemical engineers can sketch reactions. Architects do nothing but sketch.
Why? Because practice makes perfect: In most technical fields, thinking on paper is an important part of basic education.
Why? Because even at the highest level so much of the most important creative work is done on whiteboards or notebooks or lunch-counter napkins, often in the middle of an impromptu jam session with the smartest peers you can find. My Ph.D. work was mostly involved with machines and materials, but the essential plans and conclusions were sketched out on a set of whiteboards in a series of evening discussions after eating pizza with some folks from my research group. This was completely typical.
If you can't think about programming without electronic help, does that mean you can't think about programming while walking down the street, or showering, or shaving, or doing dishes? If you can't discuss software, even at the oh-so-basic level of an interview question, without a screen and keyboard in front of you, does that mean that you can't design software while eating at a nice restaurant with your collaborators?
If you can't communicate technically in person in real time -- and we're talking really basic stuff, not formal proofs of Euclid's Algorithm or anything -- why should you be hired onto a team? The team can just outsource you.
(In short, yes one should be able to think about programming problems while away from the computer. However, one doesn't arrive at a solution away from the computer. Napkins don't compile.)
Writing code in an interview will tell you one of two things:
1. This person has encountered the problem before and is simply regurgitating the solution they came to when they had four months to work on it and the assistance of a TA and other students.
2. This person has not encountered the problem before.
While the latter case is considered ideal, it only works if you know how to ask the proper questions. I suggest avoiding it because there are more direct ways to get what you want to know.
My suggestion? Code reviews. Have some pre-made examples or ask the candidate to bring their own code. No one has to be put on the spot. It's much easier to control the dialog.
PS: What would you do if someone said write a function that will sort these three numbers? Hint: there is no safe answer.
I'd be interested in hearing what you're thinking of -- but if you're right that actually makes it a perfect interview question. The baseline implementation is simple, but you can keep pointing out deficiencies and observe how the candidate attacks the revised problem.
The point of interview coding problems isn't to produces production-quality code with no preparation, it's to gain some insight in the way the candidate attacks a problem, particularly when approaching the edge of his abilities.
At one of the interviews I went through for my current job, I completely tanked on a question that fished for a particular design pattern. But while I didn't figure out I needed to use that particular pattern, I was able to participate actively in the discussion on how and why the solution I chose wasn't optimal.
I find many programmers have issues quickly context switching between thinking about the problem, discovering what the constraints are, coding a solution, and then debugging that solution. Ask them to do it on a blackboard under a little pressure and they can't do it. The problem is the better programmers often do worse in these situations because they think about more things at each step so they need to keep purging more ideas.
P.S. Let's assume, oh interviewer, that your answer to my question "what kind of numbers?" is "integers, for now".
sort(a,b,c) {
exchange(a,b) if a > b
exchange(b,c) if b > c
exchange(a,b) if a > b
return (a,b,c)
}
<?php
$sorted = sort($unsorted);
Now you could ask me to find the bug and I'd write out some test cases in the margin. Or you could ask me what happens if a string gets in there, or how I'd generalize this algorithm to N numbers, or what kind of language I'm using with this exchange primitive in it (answer: my own highly arbitrary mental pseudocode!) or how I'd implement the exchange primitive, or how fast this algorithm will run on N numbers (not very; it looks to be O(N!), and that's not counting the fact that my exchange primitive is probably wicked slow).
Or, depending on the job, you might head in some other direction.
EX1: You wrote code before asking for more details you fail.
EX2: Then again some people view asking questions as stalling.
Perhaps they are thinking about sorting a vector and want you to use the the language library of your choice or write something. Or perhaps they want a few lines of ASM that include at most 2 swaps and:
x = x xor y
y = x xor y
x = x xor y
EX2: Then again some people view asking questions as stalling.
This is interviewing, not a mathematics competition or a judicial proceeding. It's not a test, it's a first date.
If your interviewer applies either of these rules so strictly that you "fail", you probably don't want to work for them...
...unless you really are the type of person who wants a work environment where such rules apply strictly, and you happen to have the same ruleset as the person who is interviewing you. In which case the two of you may work together very well, and the interview will be a big success.
The first rule of interviewing is: Although one of you is leading, and one of you is following, you are interviewing each other. It's a conversation. If I start out answering your question on a track that you don't want to follow -- integers and not vectors, Lisp but not ASM -- you need to lead me in the right direction, just as you would do if we were discussing a real problem. Your ability to guide the conversation, and my ability to follow, and our resultant ability to establish a rapport and keep the conversation moving, is what is under test here. The problem itself is far less important.
P.S. I have seen that XOR trick before, once, but it still blows my mind. I can never understand it without spending half an hour in a corner muttering to myself. Is there a field where that trick is so idiomatic that it's a reasonable thing to expect a candidate to come up with it an interview?
PS: In the end he still he still had 12 bit's of unused memory and in his words "you can do a lot with 12 bit's of ram".
/* assume x = 5, y = 10; */
x = x + y; /* x = 5 + 10 = 15*/
y = x - y; /* y = 15 - 10 = 5 */
x = x - y; /* x = 15 - 5 = 10 */ r[0] = Min( a, Min( b, c ) )
r[2] = Max( a, Max( b, c ) )
r[1] = a + b + c - r[0] - r[2]
Of course you could always go with the boring solution:
if a > b: swap( a, b )
if b > c: swap( b, c )
if a > b: swap( a, b )Here's a homework question that I got last week:
Two numbers x and y in 0..50 with x <= y get selected. Bob gets told x^2+y^2, Alice gets told x+y. Alice and Bob have the following conversation:
- Bob: I don't know which numbers x and y are.
- Alice: I don't know which numbers x and y are.
- Bob: I don't know which numbers x and y are.
- Alice: I don't know which numbers x and y are.
- Bob: I don't know which numbers x and y are.
- Alice: I don't know which numbers x and y are.
- Bob: Now I know which numbers x and y are.
Question: which numbers can x and y be?
If you're able to solve this problem by firing up your editor and starting coding the 10 lines of code required to solve it I'd be very surprised. Hard problems require real thinking.
In Bob's stage, eliminate pairs that give a unique value for x^2 + y^2. If such a unique value matches the value Bob was given, then obviously you know x, y for sure. Otherwise, switch to Alice's stage, which operates on the remaining pairs from Bob's stage, and eliminates all pairs that give a unique value for x + y.
The weirdness in the question is both Alice and Bob are performing the exact same algorithm, but (to use CSP parlance) there is a channel from Bob to Alice in Bob's stage, and vice versa, due to the different information given to both parties. Alice must wait for Bob to declare if the algorithm has terminated or not.
The idea is that if Bob says that he doesn't know what x and y are, then Alice can deduce some things from that, because she knows for which x and y he would know what they are. For example, after the first time Bob says he doesn't know Alice can deduce that x=1 y=1 is not the solution, because then Bob would know (because Bob gets told 1^2 + 1^2 = 2, and 2 = x^2 + y^2 implies x=1 y=1).
Ok that's the part I was missing. The problem makes significantly more sense now.
The cool thing about the blue eyes puzzle is that you can solve it without a computer or pen and paper.
Why, then, do so many interview processes assume that the ability to, essentially, vomit rote-memorized bits of syntax, argument orders, etc. onto a whiteboard is so important that any "real programmer" should be able to do it on command?
See Tim Bray doesn't know operator precedence rules. It's perfectly possible to live and work and thrive without memorizing everything. This is what being an engineer is all about. Yet the FUD that goes into interview & recruitment processes (as well as other aspects of a programmer's work, but that's a longer story) is astonishing.
If you're expecting me to write out code that can be ocr'ed and compiled, which is basically what most whiteboard tests expect, I'd disagree.
I failed one whiteboard test (C#) because:
(a) I didn't know the formatting for Console.WriteLine off the top of my head, because who the hell uses that when writing an app anyway?
(b) Because I didn't check the List<int> parameter too see if it was null, even though I did handle an empty list correctly.
With the whole interview thing going on, I didn't even argue point (b), there was too much going on. On the way home I was thinking about it, and as a design decision I actually would've just let the function throw a NullPointerException. No specification was provided to the contrary. Should I have wrapped it in a FooAppException, java-style? Or just silently done nothing even though the inputs were invalid?
I wouldn't expect it to be perfect or even compilable, just the same as a widget or schematic. Physicists and mathematicians have an advantage in producing 'perfect' things on paper, because they 'solve' where the other examples you gave 'develop'. They work in the other direction; the mathematical pursuits analyze systems whereas you have asked the other pursuits to create systems, which is much more difficult on a piece of paper.
Now, if you had the designers working forwards- analyzing code or schematics or widgets- then I would expect an accurate answer (with a little fuzz for basic mistakes, same as I'd give a mathematician slack for accidentally dropping a minus sign)
There are plenty of other fields that don't work quite that way.
You wouldn't interview a baseball player by asking him to describe hitting a curve ball on paper, or a nurse by asking him to draw stiches on a piece of paper. Every field has its own medium of expression. For many fields, that's paper. For programming, it's a text editor.
College undergraduates: If you want to do well in interviews, especially ones that involve whiteboard coding, take or audit a graduate course on programming languages and pay more-than-average attention. Your code will be better, especially in rarely encountered (and thus poorly understood) corner cases. And you will interview better.
Both C++ and Java provide "dynamic dispatch" as a language feature. However, by default, C++ does not use it; by default, it uses "ad hoc polymorphism" on the static types of its arguments, including the argument passed as the implicit this pointer. In order to enable "dynamic dispatch", the member function you want it applied to must be decorated with the "virtual" keyword.
If someone is able to reason at the level of "dynamic dispatch" and "ad hoc polymorphism" and how to activate these features, s/he should be able to write correct, OO code on a whiteboard. However, if s/he is not formally trained in programming languages, while this person may understand this on some level in practice, s/he may under pressure lose this distinction in whiteboard code, particularly if there is no test environment available.
I don't honestly believe that anyone who has done OOP in C++ has never run into this, so I chalk it up to lack of formal training on programming languages. And while this person may not be appropriate for a company that requires sophistication on the very specific topic of programming languages, this person may be appropriate for another.
I have plenty more examples, but this is a simple one that I think illustrates my point for most people.
On a side note, becoming somewhat formally trained in programming languages, whether or not this is the field of computer science you are interested in, allows you to easily pick up powerful languages, such as Scala, which some people are apparently unable to pick up.
EDIT: Fixed a typo.
I do not think domain expertise should be marginalized, especially when the domain is deep. Someone who is smart and fast but who has only ever thought in Spring and Hibernate isn't going to be able to start collapsing strongly connected components in call graphs. There is far too much distance between a blank slate and being able to do that effectively.
And how can a person who is able to achieve mastery of a particularly difficult domain be unintelligent? Truly becoming a master at something is a rare trait. I read in _Coders at Work_ that Ken Thompson, the inventor of Unix, looks specifically for the ability to acquire mastery: He picks an item on the candidate's resume, whether or not he himself knows anything about it, and proceeds to grill the interviewee about it. He reasons that, if the candidate is unable to convincingly convey mastery to someone who is entirely uninitiated to the topic, then how can this person be an effective programmer and team mate (at Google)? On the other hand, if this very same tactic actually sets the candidate off -- that is, the candidate starts waxing romantic about the topic while deftly dispatching any technical concerns -- then not only will this person be effective but this is someone who will be a pleasure to work with.
BTW, I believe that knowing the terminology is of only secondary importance. Its purpose is to effectively verbalize fine distinctions to team mates and often serves as a marker for greater sophistication. (Note that plenty of people throws terms around _incorrectly_, often very much so; I am specifically not referring to these people.)
> And as much as I want to just accept it (for reasons of self-aggrandisement)
For people who identify as "great programmers", there's a huge amount of ego and status that comes with this view. Don't get me wrong, I think that computer science education in the US is pretty bad, and lots of people come out of school without having great programming skills, but there's a huge difference between the elitism of "almost everybody but me is a bozo" and the more complex reality.
Take this part for example: 'Somehow though I don't reckon there are 99 working programmers sitting there reading those posts thinking "…yeah I am a bit of a failure, I wish I was one of those 1 in a 100 dudes"'.
Well, 99 out of 100 programmers don't read those posts. That's how they get by. They never confront themselves with what they don't know, don't read tech blogs, don't visit HN and never go to conferences, user groups etcetera.
I've been working in this business for over 20 years, I have no illusions about my own limited abilities, but yeah, 99 out of 100 doesn't seem that unreal to me. The vast majority of programmers I worked with are utterly incompetent.
BTW, in interviews I prefer to let the candidate talk about code rather than writing it. Conceptual understanding, and the ability to express that already tells me more than enough. Only when it comes to juniors fresh out of college I may want some confirmation that they can do more than talk a good game.
Then again, I've been in interviews that were so brutally hard I stormed off rather than keep dealing with the issue. No exaggeration: I've been plopped down at an unfamiliar machine with only 1 putty window to a unix box (that didn't have emacs!) and told to write an ls clone in 30 minutes, without being allowed to open a web browser (and that was only one of three tasks to do in 1.5 hours)
But I also can't help but wonder if the author is in his own self-selected world. Now that I'm in startup-land, I see fewer incompetent people. Even being in Silicon Valley strongly ups your odds of meeting people who are competent. When I was interviewing outside of the Valley, things sometimes were confusingly bad.
You can argue that someone who has memorized the [Foo] standard library might be better at some things than someone who hasn't, of course. But I think that what you're testing for is at best tangental to the ability to design/write good code.
This post just seems to be emphatically reasking the question "Really, guys? Really??"
Joel actually has a pretty good theory - the same 199 incompetents plus one good programmer apply to all job openings. The good programmer gets hired and the other 199 flock to the next job opening. Occam's razor selects that explanation.
Double flame then! Someone give us some real statistics!
Also, we're not looking for exhaustive answers, eg. in the TCP/UDP case all we want to hear is guaranteed delivery, packet reordering, stream oriented, packet oriented, UDP is good for audio, that's about it.
You should know we're a startup writing a distributed database (hence the n(n-1)/2 question), so virtually all code involves fairly low-level networking, and ACID is an everyday issue. This all of course is obvious from our website, which the interviewee presumably checks. The code is open-source, so they could actually look at the code to see what kind of questions to expect.
http://mathworld.wolfram.com/TotalGraph.html http://mathworld.wolfram.com/CompleteGraph.html
Thanks, btw, I hadn't actually come across total graph before, you learn something everyday.
My first distributed networking framework (in our Keyspace product) used unidirectional connections, eg. I had two connections per node-pair (n(n-1) total). This was OK because Keyspace was meant to run on 3 nodes, and it's very easy to handle in terms of code.
However in ScalienDB, which is a generic sharded database meant to be run on 10-100s nodes, I wanted to get that /2 in the formula, after all "easy to handle in terms of code" is not a good excuse for having 2x as many TCP connections on the switch! It was surprisingly error-prone to get this right. The basic problem is both sides initiating connections, and then figuring out which one to drop.
And honestly, are you telling me that nobody answered in "however long it takes", ie. nobody just counted them on paper?
2. That's exactly what the underlying problem is; oversimplifications regarding interview processes, a complex enough human interaction as it is.
Btw., our pre-screening per-email test is to write the in-place remove_char() function which was posted here on HN a couple of months ago [10 LOC], and to write an instrusive stack [50 LOC]. People usually get them wrong but get it right the second time, when I explain what the point is.
So, that's why I ask design problems. I ask candidates to draw diagrams of databases that meet a certain need. I ask them to draw flow charts and state diagrams. These are much more conducive to human communication. I need to know that they can think before they can program. I'd hire someone with no experience in Python to start a brand new project in Python if they showed they had excellent problem solving skills because the hows of writing Python code is just going to be one more problem to solve for them. They'll figure it out and do what they need to do.
When was the last time you know everything about how to program something before you started on the project? If the answer is "ever", did you stick around for the end or quit because you were bored?
I remember reading Joel's post alluding to this back in the day, then Jeff's a couple of years ago
Hmmm. The Raganwald post that Jeff's is based on says this at the top:
If you think that I just claimed that 199 out of 200 working programmers cannot code, stop immediately and either read my follow-up explaining the math, or read Joel’s article on why companies aren’t as picky as they think they are. Thank you.
Wait, so what did Joel's post say back in the day?
That means, in this horribly simplified universe, that the entire world could consist of 1,000,000 programmers, of whom the worst 199 keep applying for every job and never getting them, but the best 999,801 always get jobs as soon as they apply for one. So every time a job is listed the 199 losers apply, as usual, and one guy from the pool of 999,801 applies, and he gets the job, of course, because he's the best, and now, in this contrived example, every employer thinks they're getting the top 0.5% when they're actually getting the top 99.9801%.
Ok, let's see how low you can go in your quest for faux controversy:
One question immediately springs to mind: Is that x out of y applicants, or x out of y working developers? There is a massive distinction.
O RLY?
All of a sudden the headline is: X out of Y Applicants for Positions Advertised By My Company Can't Program. That's a whole lot less impressive/sensational and whole lot closer to reality.
Actually, Joel's impressive/sensational headline was: News. I think you might be projecting.
The linked list question is designed as a smoke test. Admittedly it's a piece of code that one can do loads of coding without ever having to write. But bear in mind that in an interview we're looking for some code indicative of one's ability in the space of an hour, code that you actually write for production takes weeks-months-years and we simply don't have that kind of time. Questions like this have a high signal to noise; they're just esoteric enough that few people actually have a full implementation committed to memory but simple enough that anyone who gets the concepts of pointers and recursion should know just what to do.
When runs of bad candidates pop up we do indeed reinspect our process. One major improvement to the process is to ask people question their resume suggests they should be more comfortable with. Ask questions in the more esoteric languages they know, ask questions in the field they did their research on. Obviously this greatly increases the chance of correct answers, thus these type of questions have a smaller correlation between a correct answer and a good hire. But what these questions can really do well is convince you that when this person understands something, he really understands it; he gets it well enough that he can very quickly explain it to me. This is very good sign in a potential hire.
As a result, you get a wide variance of commitment, and therefore talent.
Everybody knows that this statistic refers to people applying for jobs, not working programmers. Everybody who's ever tried to hire in this industry has observed it first hand. Every article about it goes to pains to explain that it's a problem involving unemployable programmers applying for jobs they don't deserve.
I'm not sure who this author thinks he's arguing with.
EDIT: also change the list to point to the new first node.
Theres an important point here that he didn't make. The ones that aren't remotely qualified to even apply for the position? They applied to all the positions at every company they could find. Of course they seem to outnumber the remotely competent programmers; they sent out far more resumes, and by all the non-scientific metrics they get counted that many more times.
In particular, this line bothers me: 'How much time did you spend making sure your ad was sufficiently attractive to the star developers and a sufficient deterrent to the crappy ones?' Making the ad attractive to 'star developers' is not the issue here, making it less attractive to resume-spammers is.
We're only hearing from the subset of interview processes that attracted subpar candidates or had particularly bad luck.
An easy metric for suck doesn't immediately pop into my head.
But it's a good point. That said, if I were interviewing someone who was clearly a good coder but who said they didn't test their code, I would still hire them as long as they would be willing to start writing some tests.
Consider psychopaths. They are not many (in percent of the male population), but if you go out and meet guys socially, you will meet a much larger fraction of them. That is because they use people and have to get new social environments quite often...
I think it is the same when you interview programmers; the bad ones tend to apply for jobs much more. The interview process has a built in selection bias, so people doing interviews go crazy because most people they see are ass hats.
And about white boards -- there are lots of subjects where I have good experience, but not recent enough to do a white board test. OTOH, one of the smartest people I studied with aced all the math courses but could never learn to program. [Edit: What I was trying to say here, an interviewer couldn't see a difference between us two by asking about e.g. a language I haven't used in a few years.]
Disclaimer: I don't know how good I am, but I only look for work at most every few years... :-)