Enough with the Programming Puzzles
linkedin.com
linkedin.com
We don't like writing code on the whiteboard, we don't like mind teasers, we don't like programming puzzles, and so on. We complain about everything.
Well, hiring is a difficult problem, and nobody has a universal solution yet. But the truth is that if I want to get a job, I will need eat the humble pie and get out there and try to perform as good as I can to whatever I am asked. If I don't, somebody else will.
However, remember that interview is a two way process, if I don't like the employer's process, then maybe I will not even interview for them. In this case though, I cannot expect them to give me an offer, just because.
There are jobs, which have a similar hiring process. Performance artists actually need to perform to get hired but they do perform their actual work. I.e. they are not playing the anthem of Cambodia on a toy recorder in order to be hired as a death metal band drummer.
Think about it this way: they are not playing a full playlist on an actual stage with a soundcheck and live audience either. Compared to the real gig they are performing a smaller task in an artificially simplified environment. It is actually very similar to a coding interview.
People often forget that puzzles are not a thing into itself, it's a tool for discovering candidate's style and a springboard for further discussion. I'm not interested in a solution itself as much as in an ability to discuss it, explain it & improve it -- because those are the things that are being done in a workplace on a daily basis.
Do you want to work for a company that does what everyone else does without a clear well-defined rationale for it, or do you want to work for a company that has a deep understanding of what it needs from its employees, and has worked out the most efficient way to get it?
Good companies run a continuous improvement cycle on hiring practices. Bad companies do the same shit over and over and make the same mistakes again and again.
So I would be surprised if poor hiring didn't correlate with poor results in other areas.
[0] http://abovethelaw.com/2015/07/california-bar-exam-cut-from-...
1 - by social networking;
2 - on diploma.
- non programmers. And dev will likely not be found in the same social circles than those people;
- programmers. Which tend to use a technical solution to all problems.
And yes, dev are spoiled in the US, so recruiting technics move fast, trying to adapt. But in France, they are not, and we have the same old recruiting technics. And it's not better. Lot of bad dev are recruited.
When I get hired, they want me to do dumb tests.
The only difference for them is that they can stop paying me right away when I freelance, but when I'm an employee they have to pay me for at least 2 weeks to go (at least in Germany were we have 6 months probation time on new jobs, where you can be fired easily)
I made many people I worked for happy, but many people who gave me tests wouldn't even consider me as employee.
I don't know if I would have been a bad employee for the testing-companies, but I never had issues with employers or customers who hired me, so I have the feeling the companies who didn't hire me were missing out.
Plumbers, unlike computer programmers, are typically hired purely by referral or a small amount of marketing. Again, you make a terrible analogy.
A plumber got hired because
A) The risk of a bad plumber is less threatening than the leaking faucet I have right now.
B) a friend said they were a good plumber.
Since JavaScript refactoring is not as pressing as a leaking tap the hirer can take their time and subject you to a rigour out process in order to ensure that the millions of pounds of intellectual property you have access to will be handled in the correct manner.
I agree with the top voted response; the perception is engineers are beginning to whine about everything.
Not you fault, you are just one part of the sum but that sum is sounding more like a spoiled brat than ever before.
However, when I was seeking permanent employment I understood that it was not my circus and not my monkey's. I had to jump through the hoops. The same for coding as it was product management as it was for consultancy.
Rather than complain, why not invent a startup that acts as a universal badge of coding ability and is widely accepted as the industry standard for benchmarking technological ability...
The post just screams (once again) "I am a coder you should simply accept my genius and leave me in peace to work."
Law of averages just tells you that there is a bell curve of ability...at least some of the individuals on Hacker News are precisely the individuals that white boarding and puzzles are designed to catch.
Production never goes down, critical errors don't arise and time-pressure tasks are rare.
My point is; you are neither psychologist nor professional human resources executive search.
Your reasoning for not conducting testing interviews is flawed and can be challenged at every juncture precisely because this is not your area of expertise.
It's not mine either but I am not advocating sweeping policy changes across an entire professional landscape.
If you feel there is a better route; then write up, test it, get it peer reviewed and implement it.
Since you are constantly referencing the legal system you should remember it is the prosecutors role to prove the hypothesis. It is not on the defence to disprove it.
Merely rebutt sufficiently.
You have not made a single compelling argument for not testing coders with the current model.
I love all of those things. In fact, dare I say I like them more than the job itself:
(0) The given task is usually short, self-contained, and has a single best solution that can be found purely by logical reasoning.
(1) There are no requirements of the form “you have to use this obviously misdesigned technology”. All one has to do is come up with data structures and algorithms.
(2) I won't piss anyone off for trying to do things in the most elegant way possible.
What's not to love?
I suggest we work on a blackboard instead of on a whiteboard. I think they are nicer.
Would be nice.
> I suggest we work on a blackboard instead of on a whiteboard. I think they are nicer.
Using blackboard and chalk is indeed nicer. The unpleasantness comes afterwards, when you have to wash the chalk off of your clothes.
All of those are characteristics that the company might be looking for.
In short those tests aren't checking for just aptitude, they are checking for the personality characteristics of people willing to do them and tolerate the absurdity - like a cult.
I once applied as a JavaScript developer to a local company and they gave me a test in Cake PHP. This was after a face-to-face interview with the owner.
I finished most of it but their was one thing I couldn't figure out. His response via email was along the lines of " You really should learn this stuff". This was after answering a bunch of granular questions about JavaScript and passing those preliminary questions.
But I guess its my fault and I "really should learn this stuff" (j/k)
:/
If Google is asking questions about manhole covers, you better study about them before an interview. A candidate who can answer such a question would more likely want to badly work for Google. That is probably a better predictor for a good workplace performance than raw skill.
It's in no way a good predictor for being able to do good work at Google - or anywhere else.
/ha ha only serious
I reckon most jobs simply require you to not be a knob for an hour while someone asks you tricky questions about your worst behaviors. How often, day to day, are you required to explain something bad about yourself in a constructive way? Yet it is as utterly common as programming puzzles.
My supposition is that if that's where the wind is blowing, I'd had damn better face my sail that way. I hate silly problem solving puzzles as much as anyone, but not a shit lot you can do about it when they're the ones holding the keys.
The HackerRank challenge consisted of two relatively trivial multiple choice problems, a simple FizzBuzz-level coding problem, and a programming puzzle. And I do mean programming puzzle. I consider it a puzzle in the sense that it had very little to do with any classical CS domain I'm aware of (although that may just be an artifact of my own ignorance, but to someone unaware of that domain like myself, it as might as well just be a puzzle) and certainly wasn't anything one would ever encounter in front-end development (I was applying as a front-end engineer). I later found the problem online and here's a thread discussing it for those interested (the company I interviewed with wasn't Google though): http://stackoverflow.com/questions/28967020/check-if-there-e...
I finished the 2 MC questions and first coding problem within the first 5 minutes, but I wasn't able to finish the programming puzzle in the time remaining (For what it's worth, I was on the right track with my approach, but it took me a lot of trial and error to arrive at that approach, so I didn't have enough time left to finish the implementation). I got my rejection email two days later.
I really question the wisdom of using a programming puzzle like this as a filter in the first step of your interview process (or really, in any step of the interview process for that matter). You're effectively discarding candidates solely based on their inability to solve an esoteric programming puzzle under some arbitrary time constraint.
Although at the end of the day, I'm not that disappointed in this early rejection. If this programming puzzle was a sign of things to come in the next 6 steps of their interview process, I think I may have dodged a bullet. That could have been so many hours of my life that I'd never get back. Good riddance I say.
It is the opposite of what the author of the article complains about - "academic", "targeted at cs grads" - it goes against that. It is an attempt to test problem-solving ability by removing knowledge from the equation. Knowledge can then be tested separately. Fresh CS grads are not going to do better on this problem than a guy who graduated 10 years ago.
In fact, this is the kind of problem given to children (who can't be expected to have CS knowledge, and you don't want to give a big advantage to those who happen to have some) in programming competitions. I would put the difficulty at either easy problem for high schoolers or difficult problem for elementary schoolers (this doesn't mean it's super-easy, these problems have to be relatively hard or the smartest kids would all solve everything).
I am not sure why so many developers dismiss the value of this and are in fact often actively hostile to any suggestion that it might be ONE of the indicators of programming ability.
I do agree that the time constraint was probably too strict and favors people who've solved stuff like this before, mainly due to them knowing they should try to solve this with a pen and paper or in their head, then just code the solution once they've found it. Trial and error through code is a huge waste of time and this was probably your problem.
No disagreement there. I certainly wasn't trying to dismiss the ability to come up with novel solutions to problems never encountered as an indicator for programming ability and general intelligence completely. However, I certainly don't believe it's a good enough indicator to use as the sole basis for simply rejecting otherwise perfectly capable candidates before you've even talked to them.
I'm also of the opinion that interviews should strive to assess how well candidates can perform on the specific position you're hiring for (that is the main purpose of interviews, is it not?). In my own case (I was applying for a front-end engineer), and in the case of most other engineering positions, I don't believe puzzle solving is a very useful indicator for that purpose, because an overwhelming majority of engineering tasks involve applying the scientific method and engineering knowledge to solve variations of well-known engineering problems in their specific fields.
EDIT: Another point that I forgot to mention: If you are in fact interested in a candidate's ability to solve novel problems they've never encountered, at least do a better job of ensuring the problem you give them is actually novel. I found the StackOverflow thread for the problem they gave me in about 30 seconds of Googling. By not ensuring the problem novel enough that searching for it won't do you any good, you're going to be disproportionately passing dishonest candidates who would simply cheat to get through this part of your process. Certainly that can't be the result you're looking for.
Regarding the second, I agree completely. This problem is good because it doesn't give an advantage to rote memorization of basic CS from college (unlike asking about red-black trees etc), but when stolen from the internet and reused becomes bad because it gives an advantage to something worse - googling questions and memorizing answers.
I assume (or hope) Google switches up the problems they use all the time and hopefully doesn't use automated testing (because I care how someone thinks, not just whether they produce the solution in X minutes), but since you interviewed at a different company perhaps it is a cargo cult effect: Google has some of the smartest programmers and they asked this, so obviously we should also ask the same thing! Cargo culting is extremely common in IT in less-established companies, my workplace is not immune to it either. Pick things a super-successful company does that require almost zero investment, effort or true change, and copy those. It is ridiculous.
Of course, the things these companies really do right, they are too much effort, let's just copy them in superficial ways and hope for the best!
I did the online test, received the puzzle, and in the intervening period I was interviewed and offered a job by someone else. When I rang the first place to tell them the guy was very upset because it had taken so long to find someone who could pass the test!
Seems to me if you're that desperate for candidates then don't assume you're the only company in the world.
The best interviews I've had was when I was invited to come work with the team after a few meet and greets. It allows all parties to relax a little and get a taste of what working together would be like.
Typical office secretary jobs here require a work sample so you'll get a typical task for the job and have to do it on the spot (write a letter to someone, translate a text). My current job (academia) required a somewhat simple test to see if I have a decent understanding of method/statistics and the programming test was answering a couple of questions :P
In logistics you usually have to reason through a couple of typical work situations.
After entry level...it's usually just an interview and references.
I once had an agent almost freak out over the phone because he'd seen my open-source project and couldn't believe I'd created it myself. "It had to come from somewhere", I told him. "Why shouldn't it be me as much as any other random stranger on the phone?" Seems like even when I presented them with impressive, real-life work, they still didn't want to trust me :)
Except it's not, that's the problem. But this is not a matter of incompetence on the part of the employer, but of deliberate and calculated policy.
I think coding challenges provide some value, but it can be a but much to expect candidates to spend hour's of their time early in the process
At one place I work at, we're trying to get some senior-level PHP devs... and we're getting some applicants that can't even conceptually do FizzBuzz...
And since you can't have them do anything meaningful in an interview (there isn't enough time), anything you ask them to do will be a silly "programming puzzle".
The key of the exercise isn't necessarily to complete the whole task (though often people do as the exercises are interesting) but to provide a discussion piece for the code review session. Candidates are told this so that they don't feel like they have to put too much work into it (2 hours is suggested).
The code review session is key, walking through someones code and discussing decisions and trade offs. It provides a layered insight into their smarts, communication skills and thought processes. It is the closest I've gotten to actually working with someone in the interview process.
And isn't that what both parties are looking for?
A job, any job, will involve thousands of hours with at least a few hundred of them on tasks you really do not appreciate. A candidate not willing to suffer for a couple hours during the interview probably wouldn't have lasted very long anyway.
But you get paid to do those tasks. And remember that those 2 hours are an estimate, which means it'll probably take longer. Then multiply that by the number of interviews you have and you can quickly see why it becomes unfair rather than 'lazy'.
The problems themselves aren't difficult and given the time constraint, you can fit the answer in the height of one whiteboard. The trick is that you have to be fast, optimal and correct (i.e. syntax counts). From my experience, you should be able to complete each problem in about 20 minutes - i.e. a good enough solution that the interviewer moves on.
The article is right: there are books and books on doing these toy problems with array manipulation, tree traversals, sort/search and so on. If you did a TopCoder SRM every night for a few months, you would probably ace most of these interviews.
But this is the issue: the interviews basically test how well you prepared. They don't simulate a work environment and I don't think I was ever asked any serious questions about my work experience/research beyond chit-chat. Given that I was applying for a specialist computer vision role, that was a bit surprising.
In the high-end cooking world, it's not uncommon to do one of two things when looking to hire a new cook:
1) Make me a dish. Present the chef with the kitchen, and possibly a specific main ingredient(s), and have them make the best dish they can with that ingredient. Sometimes this is also under some kind of time constraint, to test pressure under fire, but doesn't have to be (great chefs might take weeks to develop a new recipe after all).
2) The "stage". The kitchen takes on the chef temporarily for a short period, where they will actually work on different portions of the line, so that the employer can see both how they perform in real time and where their strengths lie. This can be as little as a single shift, or up to a month or more at some very, very high end kitchens like Thomas Keller's. Crucially as well, this is generally paid (it'd actually be illegal otherwise), so the prospective hiree can actually afford to dedicate that kind of time.
Now some of this is similar to stuff that some tech companies do already. But I think they often miss crucial elements that makes this actually work in a way that's not painful for both parties.
The "make me a dish" method for instance, superficially resembles the infamous "sit down at our computer and code a thing" tests I've read of many times. But there are key differences. Tooling is a pain point in programming that doesn't so much exist in cooking (in fact part of this test can be making sure the prospective cook is familiar with all the basic tools). Sitting in at a foreign editor/IDE/language etc. is likely to be more stressful than is really accurate to how one can expect a new coder to perform in the real world. The simple solution is "bring your own knives." Let the coder use their own machine, or at least give them time to set up a basic version of their favorite tools.
The other thing about "make me a dish" that's missing from programming equivalents that is I think most important is creativity. "Make me a dish" isn't about just proving you can cook, it's about proving you can create. And isn't that something we want from great programmers as well? Generally when I see and read about these tasks their very proscribed: there's a specific spec/demand/puzzle, and often with the expectation it will be solved in a particular way. One test task I was given once actually even specified what data format I was supposed to use to communicate between back and front ends. What about a mini-hackathon/jam approach instead? Give me a subject or a data source and a bit of time, and see what I can come with?
The "stage" as well is something I think is underconsidered with tech companies. Companies spend thousands of dollars on hiring practices and recruiters and advertising, but they can't just do a paid trial period instead? It makes no sense. If you think you've got a promising candidate, just take them on for a couple weeks, point them at a relatively low-hanging fruit task on the issues board, and see how they do.
Alternately, maybe just accept a fucking risk now and again. If the candidate seems to know what they're talking about in the interview, maybe just hire the fucking person, and if they suck, fire them. This is the same stricture and risk 90% of hiring practices are under, and somehow the economy hasn't collapsed yet so I am gonna tentatively suggest it seems to work.
Sometimes the path of least resistance is the easier, cheaper way.
One possible solution might be to make it a remote contract. "Work on this bug/commit/sample project at your leisure, commit by no later X date". Then moonlighting applicants or even just people who don't want to risk being off the search during the trial period can still participate on the same playing field, though they would potentially still be disadvantaged compared to unemployed applicants. I don't think though that a little bias there is entirely a bad idea personally, though you could possibly argue the merits either way.
In some countries, employment laws are so crazy, that it's almost impossible to fire someone willy nilly because they "suck". You have to give them written warnings, re-"skill" them, move them to different departments, etc. It's such a pain in the ass and then they still take you to court for unfair dismissal.
I think the true deterrent is not the law, but that it is rather expensive to train someone for 2 months then have to fire him/her, and find someone else.
Interviewing someone at that point, I'd expect the employer would be more interested to see if they're a good fit for the company culture and not waste time on academic trivialities.
http://mobile.nytimes.com/2014/02/23/opinion/sunday/friedman...
So no, using a degree isn't useful for hiring engineers apparently
Which seems a fair criticism to me. Google's hiring process may be as wacky and broken as any other corp, but they're a big name, and will attract a lot of very good applicants.
So "People who got succesfully got hired at Google, and don't have a prestigious degree, do about as well as people who got hired at Google and did have a prestigious degree."
If you're on the outside looking to get a job, you'd probably want to know what percentage of Google employees had prestigous degrees, and what other resume filling achievements the non-prestigious degree crowd had before basing any life decisions on this data point.
Is using a degree useful for hiring medical doctors, mechanical engineers, actuaries? It's really hard to believe that only Computer Science as a field has a problem where it's churning out people who are technically incapable of programming to such a degree that you need to explicitly test them in interviews through "puzzles".
I know for me and interviewing programmers my personal experience is the degree doesn't matter. I've met plenty of degreed "engineers" who couldn't program for shit from schools like MIT and Harvard. I've also met some amazing programmers from those same places. I don't know what make the difference except possible love of the activity. Not sure how to asses that in an interview or whether it correlates better with outcomes.
Likewise, some of the best engineers I have met were self-taught and curious.