"Can you solve this problem for me on the whiteboard?"
unlimitednovelty.com
unlimitednovelty.com
I mean, the author is entirely right: skilled devs can write their own ticket in this market. So, um, let's start doing that.
I also continue to think that "I have code on Github" is believed to be a career enhancer not because this is actually true but because it is something that is convenient to engineers to practice. In real life, the people whose decisions matter a) often can't read code and b) almost always have better uses of their time than reading code from someone who is, statistically, not likely to be hired.
Wouldn't that depend a lot on the position you seek to fill?
The person who can't read code shouldn't be deciding the technical merits of a candidate. If that's happening, your hiring process is broken.
Furthermore, your hiring process should be designed such that you can weed out the majority long before the in-person interview occurs. Steps in the hiring process should be scheduled such that the more time consuming parts come later.
I google any candidate who has made it to an actual in-person interview. If they have a github page, I look inside to see what they've done. It helps tremendously if the candidate tells me which projects they are most proud of. What's nice about this approach is it gives us something to talk about that the candidate is passionate about.
It's only when the candidate has nothing to show off that things move towards a more "traditional" interview, with dry coding and design questions.
I also continue to think that "I have code on Github" is believed to be a career enhancer not because this is actually true but because it is something that is convenient to engineers to practice
For what it's worth...
A while back Seth Godin wrote something about resumes being irrelevant, and a lot of stuff about "personal branding," blah, blah, blah. It wasn't without controversy, although I don't remember if it was discussed here on HN or not.
Anyway, here's an anecdotal data point for ya. I took Seth's stuff to heart. Since then I've made it a point to do things like giving talks at Tri-JUG, Tri-LUG, RTP Semantic Web Meetup, etc. And I do have code on Github (and a few other places) and I do network a lot and a lot of people know who I am and what I do.
Net result? When I got laid off from my then $dayjob about 2 weeks ago, I made 2 phone calls in my car on the way home from the office, and essentially had a new job lined up within 2 hours. No resume, no interview, etc. Somebody knew my reputation and when I said "I'm available" the answer was basically "Oh hell yeah, we want you."
I did eventually send in a resume and do an interview, but it was really just a formality to satisfy their HR process.
Also, they pointedly told me that I could skip even the formality of their coding test, because "you have code online we can look at."
So yeah, my experience is, if you do the right things, and are good at what you do, you can avoid a lot of the bullshit that's usually associated with hiring.
But my point was more that I was able to shortcut a lot of the usual rigmarole that it takes to get hired. And to the extent that I did go through any of it, it was obviously just a formality and there was no pressure or whatever, like you might normally expect. It's also just a nice ego boost to know you're wanted enough to where you can lose one job and have another lined up in a few hours.
a. A shopping list and schedule
b. diagram the layout of a kitchen used for making Mexican food
c. A recipe for Mexican food
The OP’s story is mixing two different things: Questions outside of the domain and the whiteboard vs. programming. I think they’re orthogonal.
For example, what if he was asked to go into the kitchen and prepare Creme Brulée? Clearly, that would be as bad as being asked about the ingredients.
a. A shopping list and schedule b. diagram the layout of a kitchen used for making Mexican food c. A recipe for Mexican food
---------
Not if you actually knew what you were doing.
I'm no chef, but I seen a lot of cooking shows (I know, I really earned my stripes hunh?) and they never ask them to diagram anything. What they do get them to do is cook ... a lot ... and also get assessed on they way they manage their time, a team and a kitchen.
We can email some actual chefs and put money on it if you want, but I'm pretty darned sure that if you were trying to hire a really good chef and asked them to diagram stuff for you, they'd think you were weird.
Now I did have the poor judgement to watch seasons one and two of Project Runway, a reality show that is like "Survivor" for Fashion designers, and they did have to sketch their designs often, you know why? because Fashion designers actually do a lot of sketching.
So yes, lets get back to the basics ... whiteboarding has its place, but if you're trying to hire someone that is going to be building software for you 40 hours a week, there's a whole other range of things you can look into instead of spending time seeing if they can implement a doubly linked list on your dry erase board with a crappy marker, when they'll never have to screw with one while working on your software.
So yes, lets get back to the basics ... whiteboarding has its place, but if you're trying to hire someone that is going to be building software for you 40 hours a week, there's a whole other range of things you can look into instead of spending time seeing if they can implement a doubly linked list on your dry erase board with a crappy marker, when they'll never have to screw with one while working on your software.
This is, to put it politely, a “straw man.” There are zillions of whiteboards in software development offices, because whiteboarding is part of our job. Many places paint the walls in whiteboard paint so everything’s a whiteboard. Which speaks to the exact argument you made a paragraph earlier: Asking people to whiteboard things is appropriate because software developers sketch things ALL THE TIME.
Now as to doubly-linked lists, you can’t cherry-pick an inappropriate question and blame it on whiteboards. Asking someone to whiteboard a doubly-linked list has about as much value as asking them to implement it in code. It’s fine for FizzBuzz testing, but of course we don’t implement doubly linked lists 40 hours a week, so beyond a simple filter, it’s a bad question. There are good whiteboard questions, and what they all have in common is that implementing a certain idea in code would take too long and/or obscure the underlying idea with accidental complexity.
We work with Operational Transformation. It’s an algorithm that diagrams easily, but the code can be challenging to follow if you don’t start with a basic understanding. If you were interviewing for a job that involved working with collaborative editing, and you were asked to explain some ideas about OT in a text document or on a whiteboard, would you really ask for a text editor?
p.s., the crappy marker comment is ridiculous. While it was somewhat entertaining in the story, I cannot take seriously the implied claim that all companies with whiteboards give crappy markers to interviewees. Unless there is some sort of conspiracy to look for candidates who bring their own markers to interviews.
-----------------
you take me too seriously sir :)
Asking people to whiteboard things is appropriate because software developers sketch things ALL THE TIME.
---------------------------------
Right, but what do they sketch. is it code? or is it a myriad of other things like ui's, application flow. object/db schemas? etc? Developers don't spend all day whiteboarding, so why should an entire interview process consist of that?
I'm not saying that you're saying that, I'm just getting back to the central point of the original article
Now as to doubly-linked lists, you can’t cherry-pick an inappropriate question and blame it on whiteboards. Asking someone to whiteboard a doubly-linked list has about as much value as asking them to implement it in code. It’s fine for FizzBuzz testing, but of course we don’t implement doubly linked lists 40 hours a week, so beyond a simple filter, it’s a bad question
--------
We're not disagreeing on that, but in the article that is a complaint. That a CRUD web application shop (for example) is pulling you into an interview and asking about compilers and linked lists when its not related to what they do from day to day (I exaggerate with the compilers, but you get my point right?)
There are good whiteboard questions, and what they all have in common is that implementing a certain idea in code would take too long and/or obscure the underlying idea with accidental complexity.
------------------
We agree. Like I said, white boarding has its place, I just zeroed in on your chef remark because it seemed a little odd to me that you'd suggest that even chefs, who hardly every use white boards (to my knowledge) would have a use for whiteboarding/sketching.
In an (kinda) related note, I think software companies could do with a bit of humility. A good number of places I've interviewed at are disrespectful of the people who come in to interview with them and tend to treat them with disdain until they show they are clearly smarter than they are. I think theres a part of hacker culture that encourages that sort of thing, but the market is tight, and getting the best people isn't a case of just putting up a help wanted ad, and having people jump through hoops any longer, because they have other options.
I failed to impart my lack of reverence with my comment about filtering. The sick thing is, there used to be trick interviews where the interviewer would deliberately not ask the person to sit down, or whatever, and test to see whether they “take command” of the situation. The correct response to a crappy marker is probably to ask if their server infrastructure is as good as their software design tooling.
You have another fantastic point:
In an (kinda) related note, I think software companies could do with a bit of humility. A good number of places I've interviewed at are disrespectful of the people who come in to interview with them and tend to treat them with disdain until they show they are clearly smarter than they are. I think theres a part of hacker culture that encourages that sort of thing, but the market is tight, and getting the best people isn't a case of just putting up a help wanted ad, and having people jump through hoops any longer, because they have other options.
This is a negative part of some development cultures, and it cuts both ways. I have seen some arrogant shops, and some arrogant people coming in for an interview. I don’t want to lick your boots or have mine licked. If the premise isn’t “Hey, we might enjoy working together as colleagues with mutual respect,” why are we going through the exercise?
The real point I'm trying to make is just if you want to test somebody's ability to write code, give them a computer, don't make them code on a whiteboard.
If you're actually using a whiteboard to assist in visualizing something or to help explain a problem, great! But that's quite different from being asked to write down code.
http://weblog.raganwald.com/2006/07/hiring-juggler_02.html
Like I said, I think we agree on a great deal about this :-D
But, we should add a few missing details to Jim's world to make it more accurate:
1. Being a chef is one of the most lucrative jobs for a person with a Bachelors degree or less.
2. The world is saturated with recipes and good food. Further, the most impressive recipes in the world are shared freely on the Internet. Because of this, it's common for non-chefs to download the best, most hyped recipes and slap them together to make an inspiring meal or two.
3. Therefore, many non-chefs try to go after chef jobs. Pay is great, and just download recipes from the Internet. Easy money, right?
4. Furthermore, world-class restaurants are in a never-ending recipe-invention arms race, trying to out-class the common fare. To them, chefs that can just cook recipes are useless. They need chefs that can invent new, impressive recipes on the fly with whatever ingredients that are on hand.
5. Because of the success of the best restaurants, every restaurant owner believes that their restaurant is potentially world-class . Therefore, they believe that they also need a team of the best inventor chefs to craft their menu.
A quick example if you ask someone to design something in ruby (creme brulee) and they say they only know Python (Flan) and you don't say "okay show me it in Python" you're totally doing it wrong - you either have the wrong candidate or you're asking the wrong questions.
I've mostly only gotten value out of white board questions when interviewing folks in architecture or design roles (its a great way to just visually explain relationships) or for testing familiarity with algorithms. I've never used them for checking language syntax.
My best experiences in job interviews on a whiteboard have been ones where the interviewer is also on the whiteboard, writing just as much as me if not more. My worst experiences have been where the interviewer was sitting in a chair on the opposite side of the room from me.
char *c[]= {"ENTER", "NEW", "POINT", "FIRST", };
char **cp[]= {c + 3, c + 2, c + 1, c};
char ***cpp= cp;
main()
{
printf("%s", **++cpp);
printf("%s ", *--*++cpp + 3);
printf("%s", *cpp[-2] + 3);
printf("%s\n", cpp[-1][-1] + 1);
}
But mainly it was there as a point of discussion to see how you approached such code.
And like most conference room whiteboards, most of the pens were dried out. I tossed those into the trashcan.
One would hope that the expected answer was to approach it by way of promptly deleting it, replacing it with something readable, and then finding the person who wrote it and kicking them in the shin at least 8 times.
I don't ask interviewees to be able to replicate a complete algorithm in a language they don't know, but I do expect that they can implement straightforward logic and requirements in some language.
They need to follow basic standards. They need to show that they are competent programmers. They don't need to necessarily write something that will absolutely run / compile on the very first pass.
cooking creme brulee != writing code
There is no claim that cooking is equivalent to writing code. The claim is that writing code is rarely done on a whiteboard (Perhaps design, architecture, or some logic back-and-forth but rarely code.) and that interviewees shouldn't be expected to perform well at something they don't do.
I think some of the whiteboard-focus came out of places where whiteboards are/were used frequently in the development flow. Whiteboards are effective ways to communicate design and architecture, even algorithms. In the Olden Days(tm) it was not uncommon to walk through an office and see code on the boards.
I don't know exactly when we got out of the habit of explaining our code, though I imagine better tools and the Agile code-as-its-own-documentation movement had some part in it. Still, I think it is hard to argue that it is irrelevant.
CS is relevant to many coding jobs. Maybe not code monkey or some HTML jobs, but for most high-quality software developer positions one does need to understand run times, recursion, OO and command line tools to do the job effectively. If the only way you can talk about those things is on a computer, to the computer, you may be a fine lone coder but you will be ineffective as a team mate.
If this style of interviewing is so awful, then how do these companies have such great engineering teams? I'm not saying there aren't reasonable approaches here, but you must try or your essay isn't worthwhile.
It's not hard to see a parallel between Clinton's economic policies and Google's interview policies. Both get credit for outcomes that may have been caused by other factors entirely.
(1) Companies like yours (Foursquare) hire conservatively to avoid false positives. You can afford to do this because...
(2) Great orgs have selection bias. You receive so many applications that you can afford to toss out all but the best. Put bluntly: only great devs make it to your office for an interview.
(3) Whiteboards are not the only test. We both know that, while Foursquare asks people to code on whiteboards, it isn't the only component of your interview. You vet people very heavily before inviting them in.
(4) This article addresses shitty whiteboard interviews. From what I recall, you guys don't have shitty whiteboard interviews.
I'll turn the question around, actually: can you point me to a great company who only hires people through whiteboard coding?
I have a keyboard... and I know how to use it.
To take an internet-famous example, Kevin Rose went on the record to say that one of the major failings at Digg was the fact that they hired anyone who could code in PHP in the beginning, but when a PHP-based solution stopped being the answer, they were left over with a bunch of PHP developers who couldn't do anything but that.
If your mexican restaurant finds out that the most successful part of the business becomes making dessert (Mexican at first, but then onto other kinds of styles), then they would be happy to have picked someone who can actually cook something else than just Mexican food.
I think the whiteboard tests are an attempt at figuring out the capacity of the interviewee to abstract the problem instead of solving it the fastest/usual way. In my experience, the ability to abstract is very closely correlated to the ability to adapt to new situations (and code better in general)
Or they could give said chef the recipe and ask them to make it - something like crème brûlée would not be easily made just following a recipe but requires culinary knowledge to complete it. A proper coding test/exercise will still be able to weed people out without having to put them on the spot with a whiteboard.
Whether or not you can do bit twiddling is far less relevant to a PHP coding position than being able to write PHP. This is not to say that a bit twiddling problem could be added as extra credit, but you should be far more concerned with their understanding of PHP.
So, to incorporate both of your interview styles, it would be a good idea to have the chef cook Mexican food, desserts, and some other stuff rather than just cook a single Mexican dish. This would give you a more complete view of his qualities. Then a decision can be made based on a variety of data rather than a single data point.
But flexibility can be too extreme. I've seen job postings for people who are good at programming AND can photoshop good UI. To me these are such completely unrelated fields that it would be very hard to find someone who is good at both. You would probably have better luck hiring two different people for those positions.
If the question is about code give me a damn editor and I will show you what I know.
However, I think that actual coding that relates to the job in discussion is important. This doesn't contradicts the idea in the article though.
Nearly every software project begins in a room with a whiteboard, we often spend days mulling over massive whiteboard drawings with lines, boxes, psuedo code etc. But once we have finished with the whiteboard everybody goes away and the code is written by multiple developers, in parallel and the end of the week (sometimes the end of the day) it is finished and it all integrates perfectly.
Repeat the above over the lifetime of a larger system, and the hours spent expressing the ideas on the whiteboard save 10 times themselves in implementation and test.
Some developers hate this approach and see it as robbing them of their "artistic freedom", but the fact is, large scalable solutions aren't designed, developed, tested and integrated by lonewolf coders (at least not as a rule). By being up front, pooling ideas, knocking them down and arguing them out in a "public" setting I have never seen a software project fail, be late, over budget or not meet requirements (at least as far as a project can meet dynamic requirements) - we call it the House diagnostic approach - throw ideas out there, see what makes sense - only differences are the domain (not medicine) and the fact that "House" becomes the group.
As a note: I don't expect candidates to be perfect at expressing themselves, or even have any experience with white board design at all. But if they can grasp the problem domain (and they should, all the sample problems I set are really basic (design a system to collect and sample some basic census survey data i.e. design a record structure (a class, a database, a file - i really don't care), define an efficient way of accessing the data (hash table of classes, database queries, a novel file directory structure - again don't care, I want to see that they can grasp and spout ideas about), and define some algorithms to work out averages, aggregates, max, min range, sort etc. (testing basic algorithm knowledge, and they they can apply this to their solution).
The breakdown of the last group, of 400 applicants:
300 were rejected in the paper sift because they failed the paper test....(like the above, but on paper, simpler and you have as much time as you want to do it.....there is a serious lack of good talent!)
79 were failed at the first interview because they had problems explaining themselves on a problem like the one described.
18 were failed at the second interview because they were not deemed to have enough experience in computer science, mathematics or physics (or whatever domain they were supposed to be expert in)
3 passed and were offered places, and have settled right into the company.
They were so happy with me after the first year they gave me a %40 raise.
Meanwhile the worst, most excruciating, took all day and I didn't get the chance to have lunch I was promised, interview where I met with 9-11 (I lost count actually) people, and had to write code for every single one of them, ended up also being for the company that was the worst to work for, where I was surrounded by mediocre programmers and the management was absolutely incompetant.
I think if I made a graph with one axis being the amount of programming required in the interview process, and the amount of suckage at the job in the other, those experiences would show correlation.
Anecdotal of course, and I have theories as to why, but that's been my experience.
You can learn all this just by talking about a hobby.
I think that people who are trained to be programmers (e.g.: not born programmers who started when they were 12 and might not have even gone to college) don't have the innate ability to recognize this ability in others. Thus they must ask them trick questions or to program, in order to satisfy for themselves that they're good.
This is actually not a test of the candidate, but a revelation of the level of competency of the interviewer.
If I wanted to, I could make %50 of the people you send me "fail" fizz buzz, just by replicating some of the things I've seen from really bad interviewers. Speaking in a heavy accent, and when asked for clarification, just repeating the exact same (indecipherable) words, while getting visibly irritated. Or insisting that they use a language they don't prefer, or failing them for leaving off a semicolon on a line, or for not producing bug free code right off the bat because they were writing on a whiteboard rather than typing at a computer, and the acts are very different.
I've interviewed a fairly large number of people, and the vast majority of people who didn't pass the interview did so because programming wasn't what they really wanted to do, or they weren't passionate about it, or they were just unable to communicate well enough to feel like a good fit.
I used to ask questions much tougher than fizz buzz and I would sit there and watch them struggle and try to help them thru, and at the end of the day, excellent programmers struggle on those things as well, and programmers with no innate ability often did them great because they'd practiced them. Why ask someone to find a loop in a singly linked list when they're never going to do that on the job? You can't ask them to architect a distributed realtime database in an interview, but you can talk to them about it.
Further, I think that companies either treat programming as a form of craftsmanship where they're looking for great artisans to build a fine product, or they treat it as a process where they're looking for minimally competent interchangeable coders. I think it is the latter that tend to ask these questions.
And I think the trained programmers aren't aware that there's a higher level, of craftsmanship.
Its completely arbitrary, but if you asked me to write fizz buzz, maybe I should conclude that, if you're having people of that level of ability interview candidates, that your company is not operating at the high level of ability that I bring to the table, and thus this interview process is likely more arbitrary than anything, and even if I get the job, its not going to be a place I want to work. (Back in the day the problems I got were a lot harder than fizz buzz, so I can do fizz buzz on the spot, I know, but I'm not saying that you should be asking harder problems, I'm saying that this kind of quizzing is a bad sign to me.)
So, when I hear people say that %50 can't pass fizz buzz, I think it reflects more on them than their candidates. I've not seen anywhere near that level of incompetence, and I don't think my resume filter is more stringent (its probably looser, because I won't let HR types who don't know how to program filter anyone out, and I read the resumes myself.)
If this post is down voted to hell, then you also have an answer as to why I didn't give my theories in my earlier comment.
Honestly, I think the hardest part of interviewing without just forcing candidates to vomit code is simply time constraints. A 45-minute interview in which a candidate doesn't have much to say about their history pretty much requires coding to give you enough information to make a decision.
I think a much better method would be to have 3 interviewers maximum, though fewer is better, have them spend an extended period of time with the candidate (preferably all at once, not in series) and to develop a rapport with the candidate.
Sure, interviews are intimidating, and if you're having trouble getting them to open up, introducing a programming question isn't going to make it easier. A programming question with a time limit, to boot.
I've been asked a lot of programming questions in my career, and back when I worked for other people, I'd generally go on 5 interviews and get four offers and a callback. Figure 20-40 programming questions every job hunt.
When I didn't get an offer, or it was clear that the interviewer was not satisfied, it was nearly always because the interviewer did an incompetent job. Maybe %5 of the time the question just stumped me. But I've seen a lot of people with really heavy accents who get visibly upset when you ask them to repeat what they said, and when they do repeat it they use exactly the same words and don't try to explain it in a different way. Often interviewers leave out key concepts or requirements when presenting the question because they've presented it so many times they forget. I've had interviewers explain it wrong, and many times I've had to correct the interviewer because I recognized the question they were actually asking.
Now I wonder, how many of these people who fail, or "don't know what a for loop is" are really failing, or not just reflecting poor interviewing skills on the part of the person asking the question?
Its really easy to have a high failure rate when you leave out a key requirement. (and then what's the candidate going to say "you never said that"?)
I've met a lot of interviewers that really weren't competent themselves. I tend to ask them questions (though I never got cheeky enough to demand that they program for me as well) and a surprisingly high percentage of them are trained coders who know one particular area, and seem prone to judge candidates by their knowledge in that area. I didn't get a job once because I was asked about assembly language and explained that I'd only had experience with 6502, Z80 and 8080, which the interviewer thought was the same as 8088 and therefore I should be able to write code in the modern intel instruction set, in assembly. This was for a company doing a social voice web app widget!
And don't get me started on interviewers asking me to write code but who won't shut up long enough for me to finish a complete thought in my head.
If you get someone to write fizz buzz for you, and your company is developing software more complicated than fizz buzz, then you haven't really learned anything from it.
If they "can't" write fizz buzz, how did they get in the room with you in the first place? Think they spent the last several years writing software but somehow lost the ability when placed on the spot in an interview? (I'm not being sarcastic, I genuinely am curious about this.)
I think there's a lot of people with a machismo complex, certainly I've been the subject of them, who are up for the same position, or wanted it, or think that nobody measures up to them, or just delight in seeing failure, or have such a narrow view of things that if you can't write assembly in a win32 environment, you're no good writing web apps either.
I'm no fan of contrived questions or hard CS stuff in a functional engineering position, but many candidates cannot pass something like Fizzbuzz.