Why Developers Hate Coding Skills Tests and What Hiring Managers Can Do
hackernoon.com
hackernoon.com
I don't have the will, or energy for that.
I have been working in the area like for the approx. last 2 decades. If anyone wants to validate if I can really code, can put 1 hour of their time and go over my github/gitlab profiles instead of asking 1 hour of my free time to redo what I was doing professionally.
This will be counted against their paid time, but not mine. This is asking some sacrifice on my part for a process everyone knows is highly subjective, and not really doing the expected.
And more to that, while working I solve real problems of the business. I do work with colleagues. I do use as much time as needed to do research. I don't get on a dumb IDE which is not integrated to my workflow, or tooling and do puzzles with illogical limits.
I have a decade of checkins and mineable Git data.
I have solved every single problem that I've ever encountered; whether technical or human (I was a manager for a long time).
I have convinced skeptical, conservative, empirical Japanese managers to fund and support development strategies, based mostly on their confidence in me, and my Integrity.
I have hundreds of pages of articles I've written in Medium and my own (multiple) sites.
I have multiple sites that I have designed, and hundreds of pages of instruction, along with entire courses and presentations.
I can trot out a long line of personal and professional references that include CEOs, VPs, lawyers, doctors, consultants, politicians and other engineers, all of whom will sing my praises to the stars -on the record.
A big reason for that, is the reason I'm not extremely well-known (outside of a certain narrow demographic). I have done most of my engineering over the last decade in Service and open-source projects that aren't about making money; but helping others.
However, when I say this, the answer I get is "IF what you say is true..."
Why the hell would anyone make a claim like that without backup? That's crazy.
OF COURSE I can back it up. My SO Story is a mile long, with links to everything.
Damn straight, there's folks better than me at pretty much any branch of what I do, but I know few that can offer the depth of knowledge across the spectrum, like I can.
AND I CAN PROVE IT.
But you'll make your decision based on a half-hour, fifty-line binary tree test that a college student would ace better than me.
I want to see lawyers do legal briefs as part of their interview. I want to see doctors do "trial surgeries." I want to see HR managers do "how would you handle this?" tests as the ONLY variable in their hiring process, while having their portfolios ignored.
After a few months of this crap, I just gave up, and started my own gig. It's working well, but slowly.
I would have been fine, taking a low salary, if the work interested me. My retirement's set. I have a couple of companies that I can use to keep my toys going.
But good luck with the "Draw Spunky" matchbook tests. I'm sure that you'll get top-shelf talent, that way.
1. If you can do all these things, then it really won't take that long to freshen up your your white board skills so that you can solve a variety of algorithmic problems easily in code on a whiteboard. If you really want to work for at a FAANG-style place, then just divert your energy for a bit into practicing white boarding. I agree it's stupid and it's probably not worth your time... which leads me to point two.
2. Given your experience and interests, it sounds like working as a cog in a mega-corp is probably not the right career path for you. You're going to do much better in a place where your role is a much bigger part of the company, and your individual skills will have a much bigger impact.
And as far as the "other professions don't have to do this" line. I worked for a while in a career that was not software engineers and to be honest, I wish there was basic, practical proficiency tests as part of interviews in more fields. The number of professionals in my last field of work that could not do the basic requirements of the field correctly was horrifying.
I was interviewed a while back for a marquée corporation. Great outfit. Would definitely be a place that I'd like to work. I'd even have moved for it, and would have been happy to put aside my ONVIF work to work with them.
They had an HR "triage" person reach out to me to make sure that I was good for the next stage.
I sent them a link to my SO Story, along with links to specific repos that addressed the needs of the position (I have repos for multiple shipping apps on the App Store, and source for LOTS of stuff that is currently in use as industrial-strength infrastructure).
The triage guy didn't bother forwarding it, and the secondary interview was...you guessed it...a fifty-line binary tree test.
To add insult to injury, it was given in Swift, when the position was for ObjC, and the tester dinged me for not writing inscrutable, obfuscated garbage code using the nil-coalescing operator (think ternary, but worse). I deliberately try to avoid using it in a "chained" fashion, as it leaves unreadable code.
Basically, I was rejected for an ObjC job, because I didn't write ugly Swift.
All my work in open source has taught me to write extremely readable, well-documented code that can be taken over by folks; often with fewer chops than mine. I know that the outfit I was applying for takes that stuff seriously. They are a model that I like to follow.
2) You're right. I have figured out that "used to be a manager" is poison. I've had a number of times where I've made it to the techs I'd work with, and the process has stopped there. I assume that they are afraid I'd be hard to work with.
You don't last long at a Japanese company unless you are REAL GOOD at being a cog. I was excellent at it. I lasted for a long time.
I was actually looking at startups. I'd be almost ideal for a startup. I was willing to work cheap, and take risks, as my retirement was set, and I live extremely humbly. I have a huge array of skills and experience. I have management experience. I clean up real good and can get Japanese managers to commit (they make American VCs look like Santa Claus).
Ah...well...water under the bridge. More will be revealed...
They aren't looking for an absolute rockstar who believes their experience puts them in a level above all other engineers, or someone who believes they are too good to go through the process everyone else went through.
They want someone who won't spread toxicity through the small team, and your numerous accolades does nothing to convey that. However, the way you've conducted yourself so far speaks volumes.
I don't do self-promotion well. I just state the facts. Apparently, that's enough to "have an attitude."
I spent most of my career at a marquée corporation; surrounded by the finest engineers and scientists in the world, where I was NOWHERE NEAR the top of the heap, and never pretended to be so.
The simple fact that stating that I have a PROVABLE PORTFOLIO means that I "have an attitude" is a sign of how sick the industry is.
BTW: That portfolio is full of errors and mistakes I've made, as well as successes. I'd never dream of getting any work by deception. I want folks to know what they are getting; warts and all.
I know damn well that I'm not a "rockstar." I was a hiring manager for years. This is not a Dunning-Kruger binge. I'm quite aware of where I fit in the food chain.
And I guess that it's obvious. I'm done looking. I wouldn't post stuff like this if I were. Sooner or later, I'll probably be looking to hire other folks. I'm a really, really good manager. You should see some of the testimonials on my LI profile. One of my "golden rules," has always been to treat others as I would want to be treated; regardless of how I am actually treated. Being a working engineer during my entire management career helped me to have a lot of empathy (and hard to BS). I was also privileged to manage a really high-functioning team of top-shelf developers. They knew more than me; and that was great.
If things don't work out...well, I can wash dishes as well as the next guy. I don't really have as much ego tied up in this stuff as it seems. I just love tech.
They don't; what they say they want is pure PR aimed at creating an image of both fit to the perceived culture of the developer community and compliance with the law.
> then you should not be hiring in this stupid broken way that we all know doesn't work and doesn't give you those things
Of course. The fact that they continue to do despite the fact that it is well-known not to produce what their PR says they want is pretty compelling evidence that their PR is nothing more than that. But as long as enough people are willing to both play the game and believe the PR, there's no incentive to change.
Actually it does (and that's part of the whole point of the massive burnout crisis caused by these tests). By and large -- they've become so distorted (either by intentionally asking candidates to solve ridiculously hard problems; or by having these goals set, or simply having the tasks so carelessly so as to achieve the same effect) that they've basically lost any grounding in reality.
The only way to soften the blow (that is, to significantly reduce the chance of immediate rejection) is to cram -- that is, to in effect make it your part-time job (for a little while) specifically "practicing for" (that is, memorizing the expected outcomes of) these tests.
Which of course defeats the entire purpose of the "test". As one would think these companies would be smart enough to know.
Yes I know many people are ready to say, "But when Iinterview I just do this simple FizzBuzz-oid thing I came up with that anyone who claims to have a CS degree should be ale to solve." What I mean is -- the last time I had to go through this ritual -- about 50 percent of the time these tests (or simply the environment in which they were conducted) were seriously off-base in one form or another.
And in more than a few cases I wish I could forget, ridiculously so.
Further they never, build these "test" platforms to reflect how we actually do development. I took one of these tests about a month ago just for shits and giggles and I hit the first questions and they where along the lines of what does this code do. My first reaction was, hmm I don't know where is my compiler and debugger that I can set breakpoints, because if I come upon code I have never seen that is literally the first thing I do, step thru it and break it down piece by piece so I can logically map it in my mind. Now granted they where small snippets of code, but the point is, that's not the way I analyze code, I never just look at code and map it out in my mind, I step thru it, add comments and build mentally digestible maps of code. Further just to make sure this test in no way reflects anything near reality the test is timed and counts down on you, no one has ever counted down on my while I am developing like a bomb is going to explode.
And that's the point none of these interview test comes close to reflecting how developers actually do development. probably the closest I came to one that did was for a contracting group who sent me a React project with a single bug, it was completely set up, you just fired it up, started stepping thru it and fixed the bug, you then checked it in, and resolved the ticket assigned to you. It took about 5 minutes, it was literally just a sniff test to see if you could run a project, find and fix a bug. It was the first one I have seen where I thought well that was a well designed test to just see if you pass the sniff test.
Also don't you have an onsite technical interview process that you use to assess the candidates abilities which invalidates any need for online coding tests?
If you can't answer yes to these questions. Unfortunately you are part of the problem.
Of course I have to trust the candidates that they made the homework themselves, but if you don't start on a trust basis that would be a really bad starting point. There's still probation to make sure they're a good fit. I'm investing a lot more work into vetting when it's an overseas candidate compared to local, but mostly to make sure we don't put someone on the street, because I didn't do my job properly.
I do have the experience, and have the code to show. Why would I be subject to a "homework"?
This process objectively discriminate against people with enough age and experience, but don't have the time outside work. Even just not having the will to do a "homework" is an enough reason to pass.
Are you hunting for young eager engineers who can put the most to self proving, or a diverse experience base?
I am not suggesting anything against your process, but highlighting the shortcomings.
I would probably pass your interview process directly.
I've allowed myself to become old. I'm well past 40.
Most of the time, rejections come within a couple of hours of submission, so I know that a "robo-scrubber" detected eld, and bounced my résumé.
It's actually funny. I see a position advertised, where I am a 100% fit, I submit a CV, and get a bounce within an hour.
Then I see the position still being advertised weeks later.
I have considered adding the following line to the top of my LinkedIN profile:
"Let's just get this out of the way: I'm over 40. If that's a problem, then let's not waste each others' time."
I REALLY don't want to work anywhere that I'm not wanted. I have the rare privilege of being able to do more or less what I want. I love coding. I'm an outstanding manager, but I don't love it.
It's just not gotten to that part. In some cases, a decent salary would have been required in order to support a move to a higher-cost-of-living area, but it still would have probably been surprisingly low.
As a manager, I made a pretty decent salary for a long time. I would not expect that, as a coder.
I have received a few rather direct answers, and once was privileged to be accidentally CCd in an email, discussing my age.
It's illegal as hell, but hard to prove, and not worth it. If you make a big deal out of it, you blackball yourself.
This kind of behavior has been rampant in the finance industry for decades. It's still crazy as hell, with 25-year-old kids running billions of dollars worth of other people's money through virtual slot machines, while acting like drunken frat boys.
When there's money to be made, people will look the other way. Check out the industries where there's not a lot of money to be made, like social service or education. You will see people being keelhauled regularly.
More importantly, you need to look beyond yourself. I'm an old developer and I don't like to be grilled on algorithms either. But my understanding is that it works very well and that no one's ever come up with anything that's better in terms of effectiveness at scale. Giving random hiring managers discretion to hire who they want simply doesn't work if you're a desirable place to work and need some minimal level of competence across the board. What do you think would work any better? This focus on how it feels from your perspective is fine if you all you're trying to say is that it sucks for you personally but ultimately if you're trying to prove that this is a bad process for those who employ it, you need to make a case that there are better, more efficient processes. Every process will lose some qualified people - so that you may have been qualified, but were rejected isn't meaningful evidence against the process.
Btw, appeal to special credentials as a way to criticize standard procedures is an implicit argument that hiring managers should be given discretion to bypass standard procedures to hire special candidates. I've never really seen this go well, at a level below, say, industry pioneers. I do distinctly remember the one time I did that as a manager to hire someone with a rather long series of accomplishments who refused to take coding tests - I had to fire him rather quickly. The interesting part, other than the complete absence of any sort of actual productivity, is that he couldn't work with anyone, was right about everything and his long list of accomplishments was constantly used as a reason why he didn't have to do anything he didn't want to and any rule he didn't want to follow was stupid.
So I don't think standard algorithm tests are that great as a filter, but at the very least, the willingness to go through it communicates some level of willingness to do things you don't like or aren't best at. A huge part of what makes engineers successful isn't the ability to do things they are good at, but the willingness to do things they are not good at.
Suppose that I have a portfolio with...let's just say...100KLoC of extremely current (like fifteen minutes ago) code, a GitHub ID that is solid green, and over 30 public repos in one org, and over 50 in another, and some of those repos and code examples happen to fit your application EXACTLY.
Think that might tell you a bit more about me than a schoolboy test?
Asking for a friend..
Yeah...it does mean that you'll need to spend some time actually reading up on what I can do, but there would be no question at all about my abilities or relevance to your workflow (BTW: That cuts both ways. You may well decide, from looking at my stuff, that I'm not a fit).
I was a hiring manager for over twenty years, at a top-shelf corporation, and managed a really heavy-duty team. I loved long résumés. I would have KILLED for the kind of information you can get in a well-stocked SO Story.
As I said, this is just not how things are done at modern tech companies and there are good reasons - hiring managers generally don't have the authority to arbitrarily bypass standard processes because hiring managers are inherently biased towards hiring someone good enough for the time being, even if it lowers the bar. So the goal isn't to hire somebody that the hiring manager feels is good enough, but to objectively compare thousands of candidates against one another in a systematic way that keeps the bar sufficiently high enough. You should understand at least what the problem is before you can criticize the solution.
Also, generally speaking, if you pay attention to research into hiring, the two things that really stand out are that 1) general abilities are more important than specific skills and 2) standardized processes outperform ad hoc evaluation. If you're asking to be evaluated on ad hoc credentials that cannot be objectively compared to others (most people's best work cannot be shared) - you're asking people to forego processes. And that is simply not known to work very well, mainly because it's not evidence-based. Your unique set of accomplishments that cannot be compared is not known to correlate with any measure of performance because there's no way to say, people who tend to have accomplishments like yours tend to perform at a certain level.
I also don't know what you mean by "top-shelf" corporation but I'm not aware of any top software company that doesn't basically do this - your claim to have been a hiring manager at a top-shelf corporation isn't consistent with your lack of understanding of the realities faced by top-shelf corporations where there are far more candidates than qualified due to paying well above market and the need to ensure that the bar is kept consistently high, regardless of the motivations and qualities of individual hiring managers and interviewers (unless your top-shelf company isn't actually a software company, in which case it's likely not top-shelf at software).
I apologize for wasting your time.
When I ask coding questions they tend to be fairly straightforward. Do some basic transformations on data, handle a couple of edge cases (bonus points if you can see the edges of your own code), that's about it.
Don't have candidates redo kmp or a read black tree from scratch on a whiteboard in 30 minutes. That doesn't tell you much about the candidate.
On the extreme other end though, don't use interview questions that rely on a huge amount of background or domain knowledge for your specific team (unless you really need an expert asap and can't ramp them up). You can't expect candidates to burn more than a day on your company.
For example, recently I did a simple algorithmic coding test in JavaScript. The overall logic I wrote was completely correct, but there were two issues: (1) I accidentally missed a function call (which I caught later on after re-reading the code), and (2) I completely forgot about the API `Array#slice(start, end)` so had to recreate it using a loop.
When I'm doing this kind of test, often I get very dizzy and can't think straight. Once I wrote the correct code, but couldn't read the output that was printed. The interviewer remarked that the code was correct, and that's how I knew I'd not made a mistake. That is ridiculous: I was capable of writing the logic, but lost the ability to read. The only other thing which makes me feel this way, is public speaking.
I have a decade of experience, and can generally solve the problem. It's demoralising that despite years of experience, I forget the most trivial APIs/syntax. I've literally done this every day for years of my life. I have no idea how I'm meant to prepare for this, because I've clearly practiced enough, and I am losing fundamental cognitive abilities.
In general, I am much better if: nobody is watching me while I do the test, and I have enough time for my most intense nerves to pass. Or if it is a take-home coding test.
I know sometimes people wish to argue that "you should be able to code under pressure', but I was able to meet crazy deadlines at startups by working fast. There's something different about having only 30 minutes, and somebody watching you screen, judging your every move.
I have trained a lot of people and you see similar stuff with test anxiety - even if it is the easiest, open book, no pressure situation, I have had people openly weep and quit their job - the internal pressure is just too great.
But, as I said, the problem is that when I feel under the magnifying glass, basic cognitive skills like being able to read what's on the screen, or remember a syntax I've used for a decade can escape me.
If I was making mistakes on hard problems, I'd be more accepting. What I get in interviews is similar to forgetting super common words that you use every day ("thingamabob" doesn't gain the interviewer's trust).
Definitely as an interviewer you shouldn't care about syntax or remembering proper libraries. Even better, give hints about libraries, and maybe be proactive about saying you don't care about syntax.
The coding under pressure line is bullshit, code under pressure ends up being bad code that makes things worse. Rollback under pressure sure. Look at monitoring graphs under pressure. Write new code? Helllll no.
A lot of interviewers have unrealistic expectations though, it sucks.
There is no coding test which will identify the value I provide to clients. A coding test will make me look bad. However, if you instead give me a "how would you solve this" question, and I will ask questions and discuss strategies, then I will look very skilled and experienced.
So yeah, I tend to take offense when companies want to test me with a "google style" (as far as they believe) interview. Frankly, if I'm going to have to pass a Google Style interview, then I'll just interview at Google and get a better job with a higher salary than what company Y is offering.
I understand that for companies, hiring good developers is difficult. But I would argue that the supposed quality of the developers is only part of the equation, and that growing good managers is an even bigger problem. A great developer can die (or quit) under a bad manager, and a mediocre or weak developer can grow into a decent developer under a good manager.
You are asking a professional developer to write code for you. But apparently you expect them to do that for free.
In a supply constrained market, that's not how it works. If you want to do code assessments with professional developers, and therefore you don't want to do the leg work looking at and assessing their Github contributions, then you have to expect to pay them for their time.
Otherwise why should anybody who has fifteen options on their table waste their free time on you?
There was a time during the full employment era that people were paid expenses to attend interviews. It was considered good manners to respect people's time. That common courtesy seems to have gone by the wayside during the shortage of jobs era. Now the tables have turned again.
We have a bunch of notes for the interviewer with common questions the applicants ask.
The whole thing takes like an hour, and it’s not a gotcha or brain teaser. It’s more about whether you understand requirements and can work with other people.
I haven't done a lot of tests, but I have yet to do one like what you're describing.
Why can't we introduce a standardized coding skills exam where everyone is given the same problems and the same amount of time to solve them? The whole industry can collaborate and learn from each other to make a single test that gives the best hiring signal.
I'm under no illusion that such a thing would be trivial to do but it seems feasible. You get rid of the resume check, replace the recruiter call with a first-stage multiple choice exam that gates the rest of the process and replace the coding interview with a long-form exam with open questions. The long-form exams can be graded by the engineers that would otherwise be doing interviews but distributed amongst all the companies involved.
Once you've got all the technical stuff out of the way, you can do a single onsite interview for culture fit.
Seems like this is a win for everyone. Candidates can do the coding portion once instead of say 5 times, those 5 companies only have to evaluate a fifth of the candidates' exams (since the grading is shared amongst them) and you no longer have the problem of each candidate getting a completely different process to every other candidate.
Search for stories about it on HN to see how it's going.
Other than that, there is teaching to the test. You can easily argue that's what happens now, and most systems suffer for it. People learn concepts they are required to, but rarely good application of those concepts because they only have 90 minutes to cover a large topic like composition vs inheritance. You get one or two examples about animals barking and cars. Good luck figuring out how to apply that to business topics!
Isn't this problem shared by all exams? How often do people successfully cheat on their university/college entrance exams?
And hell, can't you potentially do this with a technical interview? Nobody who interviewed you is actually going to see you again and I don't recall an ID check at any of my interviews.
> Other than that, there is teaching to the test. You can easily argue that's what happens now, and most systems suffer for it.
Exactly, we already have "cracking the coding interview" and similar tools. I think this approach would be better because the questions can always be prepared for a particular year's interview from scratch, so there's no chance they've been seen before.
I don't think this is reliable. One of my interviewers became my manager; we make it a point to hire for a specific team and get someone on that team in the interview.
At my company we give candidates something to cover any expenses they've had to pay to attend an interview and take them out to lunch somewhere nice afterwards. It'd be nice to do more, but companies can't because the government has decided stopping employing immigrants is more important than doing what's right for candidates.
No one is hiding behind anything. The fact is that to pay someone properly for a half day test is not straightforward here in the UK (assuming you want to do it legally; obviously you could just give them cash). I've actually spent time looking in to it.
Wealth funges it.
Seems the problem is one of will.
Doing the dumb tests and assignments.
Also, every job ad we write explains what the hiring process is. Anyone who thinks the test is stupid presumably doesn't apply.
I'm sympathetic. But the last time I put out a job ad I got over 100 applications for a single opening. How should companies try to narrow that to a manageable number?
Just look at the CV and throw it out based on that? What do you look for on the CV that cuts out 80%+ on candidates to get the number down to an amount that I could have a short phone conversation with?
Your job opening should have had some sort of "job specification" with "essential" and "desirable" criteria. Take a quick pass over all the CVs. If they don't meet all essential criteria, then put them in a reject pile. For the remaining CVs, order them by how many of the desirable criteria they meet, then take a cut of the top n to contact. If you don't have the time to do this, then HR should. This won't take long; it's a quick yes/no. You look at the remaining CVs in detail after that initial pass and take into account their education, experience etc.
Any candidate worth their salt will already have tailored their CV for the job specification to make sure that the essential and desirable criteria are clearly shown on the CV to make your job easier. If it's not clear, then reject it. You'll have that number down to a shortlist of 5 in 60 minutes.
There's not a talent shortage in our industry nearly so much as there is a sane-hiring-practice shortage.
Maybe we should start quizzing would-be managers on these basic hiring skills before putting them in positions where they are responsible for hiring people.
Nothing worse than unsuccessfully sweating through an exercise for 20 minutes on something super simple, only to fix it within a Playground in a minute just because you get errors, warnings and autocomplete.
Recently a recruiter asked me to do a coding challenge. He told me to spend twenty minutes on it. Three hours after starting, I had completed the six questions that covered things like dynamic programming, converting between integers and Roman numerals, and processing data from csv files; and I had figured out why their submission script was causing a stack overflow and let them know about the bug.
The code was clean, it passed all but a few of the hidden test cases, and the recruiter told me I scored well; and I could get back on the work I was now behind on for the day because of my naïveté. Two weeks later, the company didn't move forward with an interview.
Why is my reaction to coding challenges an allergic one? A coding challenge (completed alone) is a cheap and wide net that is can be costly for the engineers who fall into it - and yes, I’ve already done enough of them to know something is fishy.
What if I’m about to spend my best hours of the day producing free, production-quality code, and you aren’t going to honor that with something as small as a phone call from the decision maker to talk afterward? Keep in mind, usually the discussion at this point has been: “Hello.” “Hi.” <Link to coding challenge />. These conversations don’t look very different, and, empirically, continuing doesn’t have a great expected value.
The most egregious version of this I’ve seen was an interview for an infrastructure position where the challenge was a generic coding test. That’s fantastic if you want a generic developer but it won’t tell you anything about architectural patterns or common sense. In any case, I’m just a developer and these tools are usually aimed at managers, and since this can at least give a non-technical person a hint of whether a candidate is suitable I guess that’s valuable on its own. I’d be very interested in the tech behind the product offered by this team to evaluate the candidates though...
Given a problem statement and an amount of time, write code and submit against a test suite to be scored.
Half of the tests provided zero information as to what was being tested or what failed, so one was left with a black box to guess what was wrong.
They stuck me on one side of a long table with all the other developers in the other side.
They proceeded to grill me on obscure gotchas and had me white board fucking solves for shit like palindromes.
At one point one of the developers stood up and said “look, it’s so easy!” And proceeded to write out his obviously premeditated/optimized one liner on the white board.
I didn’t know enough to respond with something like “I don’t write over optimized one liners because even I won’t remember what the fuck is going on 6 months from now”.
I remember walking back to my car so ashamed and embarrassed.
Interviews are supposed to be a prediction of your fit in the culture and ability to contribute.
In the past 10 years I’ve written lots of code, but more importantly I’ve shown up as a member of my team.
In the end, those interviews are good tests. Tests of culture within the company you’re interviewing...
I'm not sure how it would be received
Except coders get asked. One fool writes this idea down in a blog somewhere, and we're all saddled with it for a generation.
I've never given in to it. You want to hire me (and I'm always busy), you talk to a reference and make your decision.
Was that not Google? I mean I have never seen it in reality; like you, I would not entertain that kind of nonsense and no-one has ever asked me, but did this happen at scale before Google became famous for it?
I've been on both sides of this now and it's such a delicate balance.
As a developer, you want to be fairly evaluated to prove your worth.
As a hiring manager, you want to make sure you don't hire a dud (or worse, someone that will drag down the rest of your team).
I spend more than 1/2 of my day now trying to build out a team that I need to aggressively hire for. I've radically revamped our hiring questions to ensure they are:
* Based on real-world code. That means coding up components if you're a front end developer or API contracts if you're a back end developer.
* Includes a breadth of tasks. PR reviews, writing code, writing tests, and systems design are all a part of the assessment - not just algorithms tests.
* Has clear acceptance criteria. I want to remove as much bias and subjectivity from the assessment as possible. Every question we use has a Wiki page designed to formulate the question, the unit tests, and sample answers as well as grading criteria depending on the level we're hiring for. Expectations for Principal engineers are different than those for mid-level engineers.
It's not perfect. I'm not perfect. But I'm trying. SO hard. I hope that we're resolving this problem in the best way possible for both us as a company and you as the candidate.
> As a developer, you want to be fairly evaluated to prove your worth.
Indeed, but what I don't expect is to give a day or three to a potential employer knowing full well that most applications are going to be rejected. When a vacancy often gets 100+ applications it's clear people are going to be rejected for the most tenuous and marginal reasons.
If I got the job, no problem, but if not and I have another 20 applications with a day's coding hoops to jump through each time, that's an awful lot of evenings or weekends potentially being burnt...
The PR review should take 20-30 min. The phone call before the on site is 45 min. The on site is 5 hrs spread across 5 interviews and includes time for lunch
I had my reservations with the “solutions” and reasoning proposed in the article up until this point. This is a “Sorry, you are wrong” moment for me.
Much of this article is based on the premise that this is solving for hiring Senior Engineers. If you are in need of one or one is interested in your company, then there is equally high risk for the engineer. They are potentially giving up a stable job, uprooting their family, putting a job hop on their resume, etc. That’s a lot of financial risk. There’s also the risk of your company not being as good as they hoped. Yes, that comes down to negotiations, but you are no where near that stage at this point in the evaluation process.
Again, I must reiterate that this is wrong because this article is geared towards Senior Engineers. Juniors and Mids, fine. I can accept that. However, for an article that started out trying to use data from interviews to understand both perspectives, it started skewing heavily towards the employer’s side by the end.
I did have one fun test, but it wasn't live; it was a do-overnight thing, creating a URL shortener using Elixir and GenServer. I learned a lot, and I actually enjoyed solving the well-defined challenge. But that's certainly not something I've seen in a 1-2hr live coding test.
> Does it make you a better professional musician if you can play all scales perfectly at any time? Absolutely not! In fact I’d argue that you’re wasting precious practice time.
I'm not a skilled musician by any means but I have often read and believe that there really isn't anything more valuable than practicing scales. It builds your brain's muscle memory so you can play anything you imagine/feel at any moment.
But sometimes if I was asked to solved a complicated problem as a take-home test during interview, I might be hesitated to do my best because they might steal my ideas for free like this[1].
[1] https://www.reddit.com/r/jobs/comments/4tutqr/potential_empl...
What I DO dislike is ones which have failed to explain themselves clearly which will result in someone who probably wasn't a good fit for the role wasting far too much time on trying to get something to work that wasn't set up clearly enough in the spec. For this reason I think trick questions and crap like that are the absolute devil.
To me it seems like a big chunk of the reason for the tests is so there's something to ask in the interviews though. The technical parts of almost every interview I done in the last few weeks centered around the tests (in one case I had done them in such a rush I couldn't actually remember my code). I think a lot of interviewers begrudgingly support keeping the tests in their company so they don't have to become better interviewers.
Sometimes the best value to the team bring a person who doesn’t know algorithms but can e.g. foresee problems or ask “stupid” questions which lead others to think differently.
[1] https://en.wikipedia.org/w/index.php?title=Opportunity_cost&...
Writing a resume is fine, because I can copy paste it. Interviewing is fine because it's me talking to another person. They invest time, I invest time. I would even be fine with "solve this thing, print it out, bring it to the interview so we can talk about it". That would be a good alternative to a whiteboard interview.
But doing a coding test for multiple hours without any compensation, and with the very real possibility of getting ghosted for whatever reason... That is just unacceptable.
The take-home asks candidates to complete a basic installation of our product and document what they did in detail. The latter part forces candidates to provide a writing sample in the domain they'd be working in.
The one problem is that the writeup, which I'd find quite useful in evaluating a candidate, is just thrown into the bin as best I can tell. It's not available in our applicant tracking system, and HR doesn't provide it to interviewers along with other interview materials. I'm not actually sure anyone reviews it at all.
I'd also point to the companies that put a candidate alone in a room with the test and wait for the candidate to finish, again dismissing the candidate if they don't produce an answer that matches.
Suffice to say I was looking for to_string, and what I did was blow through too much of my time trying to remember this, and writing my own function to do it instead, which worked fine, but then I ran out of time and didn't finish all the tasks in the time permitted. For want of remembering to_string.
What you describe is how they should be, imo. But it's absolutely not what I have experienced.
First kind is that of translating application logic into code using the pre-established frameworks and patterns. This involves application domain understanding and modeling of domain into system concepts exposed by the application platform's underlying frameworks and components.
Second kind is building these frameworks and components and assembling them into productive platforms with right system concepts that nicely solves for the application domain.
Why these two kinds? Because the nature (volume, churn, complexity axis) of the work differs significantly across this cut so much that the background experience and skills needed by the programmer for these two are very different.
In an interview, you cannot really verify the direct skills needed for either of these two types of jobs directly. Instead you can look for proxies. What are those:
1. Programming proficiency in a programming language of candidate's choice – you can verify this with an in-person 2-hour coding round within the interview loop. Goal of this round is to ask the candidate to demonstrate his mastery over the programming language of his choice by converting a given application logic into functioning programming. You can do this by giving a programming environment (developer laptop with Internet connection) with a written down problem statement with application logic and test input/output samples. After the candidate has spent 60-90 minutes on arriving at a working solution on their own, spend another 30 minutes doing a live code review and then asking the candidate to make one or two modifications (enhancements) which they can take another hour to complete. Clearly state the evaluation criteria – completeness of functionality, test cases, readability of code (passing a basic code review) etc. For candidate it is 2.5 hours, for interviewer it is 1-hour.
2. Expertise in relevant area – if the candidate has previously worked on either the same/similar application domain or built platforms for same/similar problem space, then that can be explored through multiple whiteboard discussions focusing on specific aspects of this supposed expertise.
3. Learnability and Problem-solving – most important skill in software industry is ability to figure things out from vague undefined scenarios and learn new things needed for doing so. Problem solving is being able to reason through a set of stated and unstated conditions/constraints to arrive at an acceptable way forward in any situation. These can be assessed through interview conversations as well as reference checks, and inferred from their work experience and accomplishments.
The above skills list is in increasing order of priority and their relative importance ratio increases for more senior developers.
The problem with automated coding tests is they only test for the basic programming skill and the evaluation criteria is not nuanced and the decision is usually harsh/hard/final. This is a waste of everyone time, but it definitely far worse for the candidate. Hence any smart and reasonably senior candidate would refuse to participate in interview loops that have these automated tests.
These sorts of hiring filters are not something that can be solved for industry wide, because many developers don't want a common solution. Otherwise the solution is simple, because absolutely every other industry has solved this problem.
The solution is universal, objective, screen criteria backed by industry, which is also known as licensing. Using a test prove that developers have required knowledge, can read code (this is the big one), and can make certain basic decisions. In addition to testing also require documented experience from trusted and validated sources, continuing education, and some form of educational baseline (not necessary academic).
Absolutely every other industry has figured this out. Licensing isn't magic and doesn't create talent, but it is an excellent initial filter.
---
When looking at potential employers there are two things I look for now: subjectivity and selectivity. If the potential employer is not very about skills assessments I get nervous they are looking for the wrong qualities in applicants such that they may contain a lot of dead weight on their existing teams.
On the other hand if they are selective about skills, but that selection is purely subjective I go from feeling nervous to thinking they are phony. I get the feeling I am on a bad date that will end up as a broken marriage. The hiring party has absolutely no idea what they want, but they have some primitive notions about what they don't want. The common root cause of subjective nonsense is because the hiring party is insecure and seeks to qualify some level of comfort unrelated to the skills tested.
The reason why many developers don't want licensing is because many developers lack the skills necessary to complete such licensing criteria. They would be out of the job. This isn't true just of applicants or aspiring developers, but also true for many people performing hiring.
In a recent interview I had to prove I could read code in a language I have never written in before over the course of about 90 minutes, and it went very well. The goal was to prove I could quickly read/write original logic after getting over some basic syntax. The hiring party knew exactly what they wanted and were very clear in achievement criteria. Everything was pretty objective in that this process was a testing to see how far a candidate could get into a given problem in the time allotted and the effort that candidate would put into refactoring. All other technical qualities were ignored to reduce selective bias.
This always filters out the incompetent from the productive. If we can't survive a day with you in our environment, games over. If however, you survive a day in our environment and still want the job - game on.