Why I won't do coding tests | Hacker News Reader

Why I won't do coding tests

developingandstuff.com

69 pointsmparramon91 comments

At my place of work we have a coding test that we issue to all applicants for the software engineering roles. Here are a few responses from our point of view...

1. We know you can probably code, you've already passed a phone conversation about your coding experience, but you may not have a GitHub profile (you might not be allowed to at your current place of work, or you might have experienced harassment through GitHub), so we're not going to penalise you for that.

2. We want to see your solution to a problem that we know REALLY well. We know the pitfalls, the places people often trip up, we know how to tell if your solution is going to be maintainable over the long term. That's why we want to see your solution to our problem, not just 'portfolio' code, or even a 'bugfix'. Not to mention the fact that the bugfix should be already done by the team, finding tasks that are low enough priority that we can give them to someone outside the team is actually difficult to do.

3. Our coding test is quite long (3 hours), but roughly capped, we ask candidates not to take longer than this. This is significantly less commitment for most than taking a day off work for a day of working/pair programming/etc. We do pair program later in the interview process, but not until we're pretty sure that bringing the candidate on-site isn't a waste of their time, and ours.

I realise that there are some devs who reject the idea of coding tests, but we have found it (in the form we do it) to provide some of the highest signal to noise ratio information in the interview process, and find it to be a really good balance of time commitments on both sides. Very few candidates drop out because of it.

We want to see your solution to a problem that we know REALLY well.

Does that give you an expectation of the sorts of things that someone new to the problem will spot? If so, how do you manage that expectation - eg if the user misses something in their 3 hour test that you only spotted after using the same problem for 6 months, how can you ignore the fact they still missed something?

Missing something doesn't rule them out, nothing in the test is a hard failure. That said, the candidates that really shine are the ones who do spot the things that others don't. It allows for the really good candidates to differentiate themselves.

Mostly though, we're focusing on the assumptions that the candidate is making about the problem, and the data structures they use to solve it, we just wouldn't be able to analyse and compare to the same extent between two pieces of portfolio code from two different candidates GitHub profiles.

Sounds like the ideal candidate or at least the one most likely to succeed is someone who has seen the problem or a version of it before. Hope your other problems are in the same class.

The problem is sufficiently random and fictional that people won't have seen it specifically before, and a lot of the focus is on code readability and maintainability, and I don't think that's something you can fake having seen a problem before.

As an interviewer you shouldn't compare a candidate's answers/solutions to what you think you would have done, you should conpare them to other candidates. This is the main reason I have a standard set of questions I make sure to cover in an interview -- not because they're awesome questions but because they allow me to "callibrate" my evaluation.

Do you have any evidence, like a study, to back up those claims of effectiveness? I'm sure you can find someone claiming any sort of useless hiring process works for them. I mean, how do you even know how things would have turned out with an alternative process?

I'm not sure what you want a study for, I assume you want a study about the effectiveness of coding tests as a measure of ability, but that's not the point I was making. My points were addressing the rejection from the author of the blog post.

I feel we get a large amount of information from coding tests, but it is only a part of a larger interview process that covers communication abilities, and many other important aspects, so it's very difficult to test in isolation.

I don't have data to back up that claim of effectiveness, but I have nearly 2 years of seeing this particular hiring process (and seeing the 'borderline' test candidates come in and always fail the interviews - tests are anonymous, we only find out after interviews). Our CTO has quite a bit more experience with this process and appears to hold similar opinions based on our conversations about it in the past.

I have no idea who you are or what you do, but in my experience #2 up to this point tends to be "here is a github gist assignment which defines a simple REST interface, please write it and add some tests."

This may be useful to weed out junior candidates with ~1-2 years experience, but those of us with 10+ it's an incredible waste of time. What exactly do you* want to see? Your homework problem doesn't even come close to covering the tools I'd use to build a highly scalable and responsive application, including but not limited to specific coding practices which offer great performance, clean API design, full blown test coverage, as well as attention to server infrastructure design and careful selection of tools which will scale with the application from 10k to 10M users (or whatever other performance characteristics your app requires).

One of the more interesting interview techniques I've experienced is when they engage me in a prior/personal project and give me the chance to completely go 100% on about something I've been passionate about (maybe even sending in a code sample and doing a code/project review). It tends to be win/win because the interviewer can clearly see the true developer experience through this method without arbitrary imposed design decisions for homework-level candidates. They can typically guide the conversation towards some of the deeper gems of interest which only an experienced developer can understand and both parties can relate to at some level. This is what I feel the true essence of an interview can and should be.

By all means use a homework assignment to weed out the junior programmers, especially if their resume indicates as much. But if they say they have 10 years of experience treat them like they do and give them the benefit of a doubt with a 30-45 minute phone screen which concretely proves that.

I've written a publicly available open source database driver full of tests and documentation which is at some level an incredibly strong indicator of higher level programming skill, yet when submitted it's been overlooked and "please write a RESTful web app" is typically the answer. That is borderline insulting, why should I work for your company? This makes me question the point of submitting my GitHub profile as a resource at all.

* The general "you" as in "people with these hiring practices." I'm not picking on danpalmer specifically.

> here is a github gist assignment which defines a simple REST interface, please write it and add some tests

I've seen this many times too, but I like to think ours is a little different, and has a bit of a different focus.

Without giving away too much about our problem (we've had it leaked online before), it's essentially a simple simulation problem. With some relatively simple data structures knowledge, and some careful thought, it's possible to complete in the 3 hours we give. It's nothing about REST, nothing about any specific technologies, nothing specialised at all, because we want to see maintainable code, in a nice structure, that the rest of the team could pick up and work with easily - not the fact you know React or something.

I'll emphasise, this really only works when we know the problem really well, and can compare solutions to those from other developers. It gives us a good feel for what the passing grade should be.

We also aren't looking for things like testing tools, or deployment strategies, or infrastructure stuff, as we would typically interview about that, and it's stuff that's a little more ingrained in a team, so not something that is going to matter from day 1, it's more something that the candidate could input into the direction of in the future - their code on the other hand will run in production from day 1, so that is vitally important.

  Our coding test is quite long (3 hours) [...] This is
  significantly less commitment for most than taking a day
  off work for a day of working/pair programming/etc.

Whether that saves candidates' time depends on the stage in the recruitment pipeline when companies send the test out.

If a company sends me a 3-hour test, then the hiring manager rejects my CV because I don't have enough Foo experience, they've wasted 3 hours of my time with the test as I wouldn't have been invited to interview anyway.

we do ask for a coding exercise, but it takes from then to 20 minutes max, and it's done in the interview because we're just interested in hearing the candidate thinking about a fairly well known problem with no single correct solution (an elevator controller)

We send our 3 hour test after CV/GitHub/etc screening, and after a phone call, the next stage after it would be an on-site technical interview. I agree that sending it before the CV screen would be bad.

Next blog post: Why I am unemployed

These coding tests are actually quite an equalizer in terms of interviews: you may be ace at answering all whiteboard/phone-screen questions, but when it comes down to actually coding -- something you are being hired to do, you can't crack it. On the other end: it lets people who may not be amazing at whiteboard questions the chance to show that despite that they can get things done.

When somebody interviews for a software engineering position, the goal of the company is to as quickly as possible get an objective opinion on the person by seeing how that person compares to other candidates they have seen. This is good for the candidate too, because who wants to be subjected to an arbitrary process?

All of the alternatives suggested in the article are fine, but they don't make the interview process very objective.

Things you can see during a coding test:

- How they tend to structure their code

- Do they like to over-engineer problems

- Do they dive right in or think things out

- How easy is their code to read?

- Are they experimental, do things in an interesting way?

- Are they prone to making bizarre design decisions

etc. etc.

The author's position is sound for the hypothetical he discussed, where, if a company starts the conversation pursuing a candidate, it's reasonable for the candidate to expect that the company has done their homework (why else would the company try to hire you in the first place?).

So I'd suppose, if the company already knows you can solve problems with software, what they don't know is how well you'll fit in with the staff they envision you working alongside of. Skipping right to the coming-in-and-collaborating step makes sense here for both sides.

The tests definitely make sense for the other main case here, where the developer starts the conversation. Then no one should expect the company staff to go out on their own and search out Random Developer #45's blog posts.

I'm curious, how do you/coworkers supervise the coding tests you have candidates perform? You can tell some things about approach and tactics from a final draft of a solution, but I would imagine most of your listed desired goals would come from watching and interacting with the candidate while the test is being performed. And in that case, it'd still seem to be a big effort for the engineers and such who need to support?

There is barely any supervision: they are given a problem, and asked for a solution which matches certain criteria. You can't find the solution on Github. What's important is that it isn't a test of someone's intelligence or personality, but rather their pragmatism and ability to actually produce code. It really depends on what values you are looking for, though.

An example problem is: parse this non-standard log file and report back some numbers/answer these questions (e.g. show a visitor funnel, based on uniques, etc.). The more encapsulated and defined the problem, the better. It's not a theory type question, such as build a btree search of this data: it's something which you could face while working at the company and you can solve it however you want. Just like in real life, there are a range of solutions and edge cases. There is a soft time limit of a few hours, but nobody is counting. However, that time limit is what changes the difficulty.

They can either use their own machine or a company machine, and any language they want (with some recommendations). They can of course ask questions anytime they want, but other than that they are left on their own.

At the end, they are asked to walk through the solution and tell us why they chose to do what they did, what issues they had, if they see alternative ways to do something, how they might change it if various conditions changed, etc. It's the opening to a conversation.

Afterwards we rarely try to actually run the program, but different people on the team are asked what they think of the solution (kind of like a code review). Since everybody has seen it many times before, there is general consensus on red flags and everybody more or less has an idea of how a particular solution fits in with other ones we've seen, but sometimes there is debate on merit or style.

So it isn't that time consuming from our point of view.

Any time I see one of these rants I can't help but assume that the author has never been remotely involved with hiring from the other end. The sheer volume of crap resumes you get from people that can't code is utterly astounding, especially from recruiters. A simple blanket "Yeah sure here do this test first" is necessary to stop your engineers drowning, though even then you still get a whole bunch of recruiter forwarded candidates that simply cheat on any test you give them, automated phone or otherwise.

That said, if a company is actively trying to poach someone, asking them to do a coding test is rather stupid, since you wouldn't be going after them if you didn't know they were good.

This. It's utterly amazing to me just how shallow the talent pool is among the totally unfiltered active job-seeker population.

As I've been told by the person looking at resumes where I work, even decent looking resumes are not a clear sign that they're particularly good.

The sheer volume of crap resumes you get from people that can't code is utterly astounding, especially from recruiters.

But he didn't suggest that people evaluate his abilities based on his resume. His post didn't even mention the words "resume" or "CV".

Any time I see one of these rants I can't help but assume ...

Maybe you should be less assuming, then.

These discussions are always frustrating, because both sides have perfectly sensible points.

From the employers' side: They really really really do get many candidates who cannot program at all. I've been on the hiring side and seen the applications, and the amount of worthless non-programmers out there is high. And it's not just for entry-level posts -- plenty of these people have a decade of experience and titles with "Senior" in them. A simple "can you write code" filter is absolutely essential for a hiring manager not to spend all their time interviewing terrible junk candidates.

From the developer's side: If they're any good at all, they really really really can get hired by a company they want to work for very quickly and with little effort, and so their motivation to jump through hoops is going to be low. If ten companies want to hire me, and two of them want me to spend half a day doing coding problems, well, guess who I'm going to get back to last/never.

The employers aren't stupid. The devs being interviewed aren't stupid. Everyone has totally sensible priorities.

So this really just comes down to individual situations. If you as an employer are having a hard time finding people and you're requiring a coding test... maybe quit requiring it, see if you get a broader applicant pool. If you as a developer are having a hard time finding a job and are rejecting employers that require a test... well, suck it up and do some of the tests.

But if a developer is happily employed or an employer is able to find good devs, well, maybe you've hit a good compromise point for now, but be willing to re-evaluate in the future.

This is the most sensible thing I've ever read on the internet.

It seems to me that there's a high-end piece of the market where none of this applies.

If you're really good, you can go your whole career without ever having to throw your resume over the wall and wade through this sort of filtering process. And if you have an experienced team who have worked with lots of good devs in the past, you can reach out directly to them and go straight to the hiring part of the interview.

It's only where those two ends have to interface with the rest of the hiring universe that you see friction like the author describes. If you're the guy who built Big Thing X and some random company tries to get you to spend 3 hours on their coding test, you're going to say no and they're never going to find anybody half as good as you. And if you're a team of Good Devs at a Genuinely Good Shop trying to hire with a job ad, you're going to get a thousand applicants who can't FizzBuzz and you'll develop scars in the form of coding tests.

I think the key as a developer is to rapidly get one's self into that category where you don't need to deal with this crap. The last time anybody dared hand me a coding test was a good fifteen years ago, and that was the last they ever heard from me.

I bet I'm nowhere near alone in that department.

Where I'm at now, I kind of skipped doing their customary coding test, because someone else vetted my competence.

Now that I think about it, having observed some discussions about other candidates' solutions, I don't know if I would have passed.

I can bring plenty of value to companies because I have broad knowledge, good taste, and devotion to product quality... but sometimes my first draft of a solution to an algorithmic problem is a little strange, or simply doesn't match the tastes of other reviewers for whatever reasons.

So I'm pretty skeptical about these things. They seem to me like a bad way to effectivize the process of judging someone's competence. I would rather have a real conversation about what they've done, and try to collaborate with them for a while.

If I can't find a candidate with whom I can have a good conversation and whom I trust without having them solve a puzzle, then I'd rather not hire anybody.

Yeah... I'm suspicious of involved coding tests. Low-level stuff (e.g. FizzBuzz) to filter out completely inappropriate applicants is one thing. But judging people on their first pass at a problem could have many pitfalls. For example, it's common to make a first pass solution that does things "weirdly" or is "over-engineered" because you want to fully explore the problem the first time, rough it out, then make a robust final version later.

Being able to quickly solve mid-tier problems is nice, but how well does it correlate with being able to solve real problems in a suitable timeframe? I don't think anyone knows.

From my experience I kind of disagree that you can weed out non-programmers by looking at their Github profiles. This might work for people that have a lot of projects/contributions there, but it doesn't work for most Github users which have (if at all) only a few stargazers and no very active projects at all. So if we'd use Github data to weed out people based on their repos/stargazers count we would probably miss a lot of very good programmers that just don't happen to be active there.

A simple coding test (ideally in a live environment over Skype) on the other hand allows me to (very roughly) judge a programmer's general ability within 10-15 minutes: Does he/she know the standard libraries of the given programming language? How does he/she structure the code?

Simple problems that are roughly related to your field of work are suited best for this from my experience. For example, as we work a lot with graph data, one task we often use is to load a set of lists containing vertices and edges from a JSON file and convert that into a linked graph structure within Python.

When hiring senior people most of this is usually not applicable as you can rely more on recommendations and the previous work experience of the candidate, which should give you better information than Github profiles / live coding tests.

Try hiring and getting flooded by applicants who can hardly implement `min`. It would be a waste of the company's time to bring all those applicants in. So long as they're not asking for a ton of my time, I don't mind doing a simple screening.

Oh that's easy (joke)...

    int min(int a, int b) {
       ArrayList<Integer> numbers = new ArrayList<Integer>();
       numbers.add(a);
       numbers.add(b);
       Collections.sort(numbers);
       return numbers.get(0);
    }

Can I suggest that you write your values to SQL and then use SQL MIN - don't want to reinvent the wheel.

Alternatively, maybe you want a big data min with Hadoop? How many things were we working with anyway, are they definitely numbers?

Yup. My test for this kind of thing is to ask people to write a function that can sum the numbers between two numbers. It's simple, touches on enough to show if the dev is at least minimally competent, and it's a bonus is they give the O(1) solution.

It's surprising how many timewasters even a simple test like that can weed out.

If and when people ask me this (exact) question I like to tell them that although I can implement the O(1) solution, I just copied it off of a 14 year old.

Seperates those who remember their math from those who don't! Brilliant. I love the Gauss equation for this (had to look it up, I need to review math more).

A totally acceptable answer. :-)

Make sure you ask for the O(1) solution, because if someone's production instincts are to write

    (fn [l h]
      (* (dec (- h l))
         (/ (+ h l) 2)))

whenever they need to sum between two numbers, then I'd argue they're a worse programmer than someone who just writes `(reduce + (range (inc l) h))`. Programmer time is expensive, and linear time is, for 99% of ranges most devs will encounter, just fine.

That is very much dependent on the circumstances. Besides, this is pre-interview weeding question, and if they give the O(1) answer, that gives use something to talk about in the interview.

Why don't you put it up in your job posting? "MinCorp is looking for developers with 5+ years of min-implementing experience. Must haves: - Must be able to implement min in 2 different languages."

It would save you tons of trouble, time AND money.

Just be ready for copy+paste from Stockoverflow.

See, now you know that the applicant knows the basic skill set AND is a quick learner.

Pfft. I just use the 'min' package on NPM.

At my last job, I was hire #3 and skipped the entire interview process (had worked with my boss for a couple of years already, so that was interview enough). As we ramped up the staff to 20+ we added rigorous interviews until it got to the point where I'm not sure I could have passed it, despite being one of the most productive members of my team in real engineering situations.

Similarly at my new company, our coding test is geared toward people with different responsibilities from mine. I'm very much big data / Spark, whereas the test is geared toward full stack developers. They skipped it for me, which is good -- I doubt I'd have passed it.

All this is to say, I am skeptical of code tests. I've seen some value (when we filter out really bad apples who can talk well but can't code) but I've seen a lot of negatives, too.

Yeah, the company I work for uses a coding test. It's a simple CLI program that is relevant to our product, business analytics, that we ask them to spend no more than 2-3 hours on. The test has several tasks to complete, we make it very clear that someone who completes only 1 task but has excellent code structure, unit tests etc etc is more likely to get the job than someone who just blitzes through all the tasks.

The task avoids assuming any knowledge of algorithms / advanced maths / databases or web frameworks. We're only interested in their approach to solve a problem with code, not what libraries they've spent years learning, or what project euler problems they've committed to memory.

I came up with the test after I was going round a job fair (for the free stuff) and was handed a coding test to produce a simple CLI twitter clone. I had no interest in applying for the job but I did the coding test anyway. It's only 2 hours of my time and I find coding / solving problems fun.

It's really helped us distinguish developers who see work as a 9-5 job that they only do because they must and those developers who just love solving problems. Maybe the technical skills of those 2 developers are roughly equal, but one of them is going to be far more valuable long term than the other. So far we've only hired one candidate after introducing the test, and they've been the best hire we've made.

As a side note, we always do a phone interview first so that we don't waste time (ours and theirs) giving the test to complete no-hopers.

> It's really helped us distinguish developers who see work as a 9-5 job that they only do because they must and those developers who just love solving problems.

The two aren't mutually exclusive. Many (most, even) of the best developers I know will come to work at 9am, be super-productive for 8 hours, and then completely switch out of Developer Mode at 5pm. It's entirely feasible for somebody to be extremely passionate about programming and learning and problem solving and all the other things that we associate with "good" developers without them doing any sort of work in their own time.

Developers are people, too. They have families, friends, hobbies, and other things that are more important than your 2-3 hour coding test. You are alienating a large number of people by asserting that the ones who complete your coding test are "distinguished". You are also seriously reducing the size of your talent pool whenever you want to hire somebody.

I think you have misunderstood me, or I wasn't clear enough. We're not looking for people who work endless overtime or spent every waking moment on open source projects.

The text you quoted says "a 9-5 job that they only do because they must", which I believe would exclude anyone "extremely passionate about programming and learning and problem solving". I'm talking about people that have no passion for it, regardless of what their other passions are.

Our new hire arrives at work at 9:30am and leaves at 5pm.

With regards to hobbies, families etc. Thats understandable if they don't have time to complete the test even though they are job hunting. However, I'd rather miss out on a good hire than risk a bad hire. My company can more easily survive spending a bit longer recruiting in our reduced talent pool compared to living with several bad hires.

We had a candidate who graduated from an Ivy League school who refused to take our code test with: "I went to ____, I shouldn't have to do this." We would have probably considered him if he asked "Can't you just look at my Github" like this person but as it stood we rejected him outright because someone with that attitude is not likely a cultural fit for us.

However, from the article point for point...

"I know how to code, and can show it" -- we don't have the time to read your entire blog and all your GitHub commits. We have hundreds of applicants.

"I have a day job, and several side projects: I won't spend a sizable chunk of my free time so they can tick some boxes about my coding skills." -- our software engineers have a job too, they can't spend several extra hours per candidate (when there may be hundreds) to look over your stuff. Second, the time it will take you to do this is WAY less than driving to the office for any of the other suggestions.

"work with them on the real job, and see how it feels" -- again, does not scale at all.

Maybe he has never been on the other side of the table. But we reject 48/50 people. Many of whom couldn't even do Fizz Buzz. The tech test is a way to make the process scale and make it so we don't waste time. If each of those 48 people cost even one hour of engineering time we just lost thousands of dollars.

Edit: on second thought, he did say the company reached out to him. We do tend to do things differently if we reach out specifically to a person. But we do that very rarely.

"we don't have the time to read your entire blog and all your GitHub commits. We have hundreds of applicants."

Applicants are expected to take their time to submit an application that stands from the rest, and bla bla bla, but employer can't be expected to visit a github profile, nice.

That's not what I said.

Employers can be expected to look at Github and websites and all that, for people who pass reasonable screenings. Not any person who emails us a resume.

A well designed code challenge takes less than an hour, which is what most candidates would take just to get to the office (one way!).

I haven't looked for a job in a while but last time I did, I spent an average of EIGHT hours per company. I would have loved to have a two hour test I can take and be immediately skip four hours in the office asking me about algorithms.

To think otherwise is some self entitled bull shit. Having a job is something YOU work for. The company gives you great benefits, high pay, training, advancement opportunities, and meaningful work. If you can't be bothered to do a 1 or 2 hour tech challenge then you don't deserve a good job.

Our side of the equation as employers, is we have the money to pay you, we work hard to build a great place for you to work, and we offer opportunities for great people to thrive. That's our investment. We spent tens of thousands or more on that before you even submitted your resume.

I don't want to make assumptions and I could very well be wrong but you sound like someone who has never read resumes. Most candidates can't even be bothered to write a cover letter. They blanket dozens of job listing with their resume. So that person who blanketed dozens of listings with their resume has a right to take up engineering time to review their Github?

One of our screening questions is "have you installed our app on your phone?" and over half of people say "no" and another quarter lied. So we're expected to spend hours reviewing your code but you can't even install our app on your phone and research the company you applied to?

Guess, what, for those people who didn't even bother to look at our app. Congratulations, you don't have to do the code challenge! You saved yourself two hours... and lost the job... but that's not what's important. You don't have to write code!

And you know what... I have fifteen years coding experience, 7 years management with hands on coding, a computer science degree from a great school, a couple high-profile names on my resume, and a screaming newborn baby and I wouldn't hesitate two seconds to spend two hours to do the silly code test for a job I want.

I don't like coding tests, but they are far better than the alternative: these typical multi-hour interviews involving lots of white-boarding fake code.

A venture-funded company who's entire technology stack is a basic Rails CRUD app grills you (in a room, or over the phone) about hypothetical uses of CS fundamentals that none of their own employees have touched since college.

I'd always rather establish my competency by submitting real working software I've written or solving some trivial coding challenges, rather than trying to solve these random brainteaser problems live in a high-stress, timed situation.

The prevalence of these tests and the emphasis employers place on them should really put to rest that nonsense about "your Github profile is your CV" once and for all.

Recruiters are generally non-technical and unable to judge the quality of your open source work, and employers generally don't give it more than a cursory glance. And while Github profiles are needlessly abstruse, I doubt even a portfolio specially curated for their viewing would fare much better. Employers just don't care; they know that it's a buyer's market for talent and that there are more than enough applicants willing to subject themselves to their interview process, however abusive or insulting it may be.

Since almost no one ever gets a job because of their open source work, why do employers still make such a big deal out of it, with some even going so far as to state that they won't look at a candidate without a Github profile? Because it shows "passion," and a "passionate" developer is one they think they can more easily take advantage of. That's it.

I think that, all other things being equal, you're likely to get a better offer with a solid github profile though.

But only if all other things indeed are equal. It most likely can't hurt, but if you start leaning on it in anyway, assuming that it can replace making an effort on your actual CV, or developing even the faintest teint of entitlement around what's expected of you in a recruitment process, it can very quickly become a liability.

If you're so obviously awesome that you can expect to special-case the recruitment process (because of your GitHub profile or celebrity status or something else), you'll generally know it. If you're uncertain, you probably can't.

I've seen a good number of GitHub profiles in a recruitment capacity in my time. Most of them are fairly trivial -- a personal blog engine, small tools, bit and pieces, vim/emacs config files, forks of popular repos with small edits, more often than not cosmetic. Sometimes small fixes PR'd back. Inevitably, presented with little-to-no context.

Rarely are the repos 'live', so I can discover how the owner grows a project over time, they are often single (or few) commit code dumps.

Only extremely rarely have I seen a project of any complexity (written by the applicant) that could serve a the basis for deeper discussion. At least once has the applicant apologised for not being able to discuss it very deeply, because it was a few years ago.

It's interesting that engineers (sellers of talent) and employers (buyers of talent) both seem to think that the other side has it easier. I don't know if there's ever been a better time to be top talent in software.

If that were true, the wages for software engineers would be going up. They aren't. http://www.epi.org/publication/pm195-stem-labor-shortages-mi...

Top talent doesn't drive average wage growth very much. My comment was specific to top talent.

Depends where you are, obviously. Good javascript devs were on £30k a few years ago. Now it's £50k.

We've had javascript dev positions open for over a month now, £25k to £55k.

...if any javascript devs are after a job in Brighton, UK, shout.

Code tests are:

1. Intrinsically unbiased.

2. Applicable for remote candidates.

3. Easily verifiable (unlike a github repo)

I'd take code tests over alternative interviewing methods any day.

Generally I agree, but on verification tests can make someone initially appear stronger than they are. E.g. if they find a stackoverflow post that matches the question.

There is also at least one "learn to code" bootcamp with a database of tests on github.

"if they find a stackoverflow post that matches the question."

To be fair, "figure out if someone else has solved this already on stackoverflow" isn't the worst approach to solving (some) problems.

I just did a coding test where the exercise was trivial, and indeed there are lots of good answers on stackoverflow. I am guessing the reason I have another interview coming, though, is that I made it easy to read, explained my logic in comments, and wrote thorough unit tests checking edge cases.

Agreed. I've seen people document forum posts they sourced as well, which I appreciate.

If I am already employed or not desperate for work, I will not do your coding challenge unless you pay me for it. Why? Because you get most (all?) of the benefits of the exercise but I need to do all the effort. Actors do get payed for doing castings.

This. Interviewing is a two-way street. As a candidate, the only information that I get from a coding challenge is that the company does not respect my free time.

> Actors do get payed for doing castings

i'm not sure where you are getting your information from, but in the real film & TV industry here in Los Angeles/Hollywood, this is simply not true.

actors do NOT get paid for doing auditions or casting calls. in some extreme circumstances -- let's say when a large amount of travel time is required to get to the audition -- some form of compensation might be able to be negotiated, but this is very, very, very rare.

source for my information: i co-ran a production company here in Hollywood CA from 2005-2011.

In Argentina they do!

An X hr coding test is very different than an X hour interview.

The interview process is intended to be a bi-directional flow of information. Giving someone a X hour programming test means that for X hours, the company gets to learn about the developer, but the developer doesn't really learn about the company.

The company is the one paying salary, so they set the terms of the interaction. But the feel of disrespect for the developers time is real and valid. If you chose to do a coding test, do so respectfully to avoid insulting the person you potentially want to hire.

I take coding tests when they require up to 3 hours of my time. Once I had a company send me a coding test that they said should not require more than 20 hours. And that was after a 20 minutes talk with the hiring manager. My first convo with anyone on the team.

Asking me to put 20 hours before even meeting the team is laughable. But, seriously, 2 hours is fine. It is fair since all candidates get the same test, plus it gives something to talk about during in-person interview and help avoid the f-ing coding at the whiteboard.

As a company interviewing candidates, we see the recruitment process much like a conversion funnel for any website: there will be a lot of hits on our spec, some people will apply, of which some will do the telephone interview, of which some will get through to the coding test, and so on.

If OP doesn't want to do a test, then he will just end up as one of the ones that didn't make it through the funnel. This then boils down to whether there are companies out there that will take him on his project's merit or whether OP is desperate for a job.

A couple points I disagree with:

* "My main gripe with coding tests is that they ask me for an investment of my time and resources so that they can gather information about me, but I'm getting nothing back." -- you can potentially progress to the next interview stage, you will refresh your memory on coding tests, etc. There is still something to be gained.

* "I've already expressed interest in their position." -- there is minimal effort required to apply for a job. As an interview we get swamped by CVs that all look equally similar - we often need something to distinguish between these CVs. We have seen a wide range of coding test submissions and from this you can glean how interested the candidate is.

Finally, while the OP's suggestions are sensible, they are not always practical for companies, especially when it scales. We can't afford to be distracted by bringing a candidate into the office and work with us especially if we have multiple candidates. Even if the company decided to bring someone in, and assuming the candidate is capable, it will take them the better of a day to get up to speed and running -- setting up the dev env, getting the source, building it, making sense of the application, making sense of the feature/bugfix, making sense of the coding style, etc. They won't be effective enough to actually check in working code.

Getting to the next interview stage is not 'getting something back' according to his criteria - at that stage he doesn't know if he wants to progress, and doing a test doesn't provide more information. It gives the company information on whether it wants to progress, but not the candidate.

Also, refreshing memory on coding tests is not exactly getting a lot back. You don't need to do an interview to do a coding test, if that's your fetish.

Not that I completely agree with the guy. I don't mind coding tests or assignments, except when I have no idea whether or not I actually want to work at a company when they give me one. So, in that case, unless I'm really bored or feeling especially unemployed, I don't put a lot of effort in. I do understand why companies do them, and the company I work for does.

Thanks, that's essentially the answer I was about to write.

I think the main disagreement here is that OP considers hiring him to be a privilege where parent considers being hired by him to be one. As long as parent is OK missing out on all the developers that aren't hungry enough to subject themselves to one-sided interviews like this, he'll be fine.

I like to think there's a high correlation between one's skills and unwillingness to accommodate arbitrary processes, but have nothing to back that up.

Speaking as a hiring manager, I won't even TOUCH a technical candidate without a coding test. Nope, burned too many times with resumes that were utterly bogus. Not wasting my time. Ideally administered at the whiteboard in front of me, so I know you didn't pay someone $5 on a chat board to do it for you.

It's not a big one - primary goal is to validate that you've actually worked with a database before. Relevant to the job.

But seriously....

- The "Oracle Certified SQL Developer" who fessed up that she didn't know SQL (couldn't even write a basic query)

- People that couldn't identify how to join tables...

- The "Google / Motorola" Python Dev who didn't understand the difference between a list and dict (which is basic to the language, very first chapter of Dive Into Python)

- And massive inattention to detail (leaving out entire sections of the request in their query)

Someone else can go attempt to manage them. I'm busy....

I have also been frustrated and surprised by the technical interview process. For a Google interview, they had me cmd+tab-ing between a google hangout and a google doc (for the code). Why isn't there something simple and free that puts these two things together?

So I took the approach of trying to improve it by creating a small side-project that is just a simple collaborative code editor and WebRTC video/audio connection.

One-click deploy to Heroku, open-source, MIT licensed.

https://github.com/crcastle/collaborative-code-conference#re...

Feedback welcome!

I really think this madness will only end if we all refuse to do coding tests, subject ourselves to multi-day interviews, etc. I do understand that this is easier to say when I have a job.

My previous company had what I think was a good hiring process: they required about 500 lines of Python code (it was a mainly django shop), and if your code was good, they'd ask you to go there one morning/afternoon and implement some minor feature in the development codebase.

For frontenders, the process was similar, but they'd ask you for some mix of JS/HTML/CSS.

The 500 was a somewhat arbitrary number, but it was basically "show us a good amount of what you have done". In my personal case, I sent them a link to one of my python libs in github, but some people without github sent some files like headers and implementation (like one or two guys for a ObjC role).

If you passed the initial code review, survived the day with the team, they would still ask for one or two contacts of previous co-workers, teachers, etc., just to have a chat. In the end, we had a very very good team there!

"I won't spend a sizable chunk of my free time so they can tick some boxes about my coding skills..." but you don't mind spending an entire day (for which you presumably took a day off work, if you're currently employed) doing it at their place of business?

I came here to say this.

Also, as an Australian who recently went to the US to do programmer job interviews, I was shocked at how widespread the idea of "whole-day interviews" were. I find it overly self-important and inappropriate when companies ask potential candidates to take an entire day off their existing jobs to come in and jump through hoops for them.

You can fire someone quite easily in the probation period if they don't perform as expected. So if after a phone screening test and a 2 hour in-office coding challenge a company can't determine a developer's worth then there's something wrong in its entire hiring process altogether.

/rant

It is strange how he mentions that one of their goals might be:

2 Only get the developers that are interested enough to perform their test in a ordered and timely manner.

and then dismiss it so lightly:

I've already expressed interest in their position. I have a day job, and several side projects: I won't spend a sizable chunk of my free time so they can tick some boxes about my coding skills.

Maybe they are not interested in people who have so little motivation.

When I want a full-time employment, companies get crazy about who they hire.

When I want a consulting gig, companies seem to give money to whoever comes first.

In the end it seems like getting an employment is a bad thing to do, since there is no real benefit of having a secure job for an engineer, because everyone is willing to pay her money for consulting stuff anyway.

Feedback from Reddit:

https://www.reddit.com/r/programming/comments/4daew9/why_i_w...

So far spending a day (or two) onsite and working on something together with the team has been the best option for me. You: - get to see what people are, how they work and what they do They: - learn whatever they need about your skills

So it's a win-win.

It may not help that the site for your cited example of a working app is returning an error:

http://www.coverr.me/

Though perhaps it's only meant for light load and HN is overwhelming it.

Yeah, that app is decommissioned. I'll update that working app soon, thanks for reminding me about it! :)

...and why you won't get our job :-)

I think it's pretty clear he'd consider your job not worth the effort - that's his entire point, isn't it?

Exactly.

On the plus side, the really good companies who eschew shitty coding tests become even more attractive to good candidates.

If you are going to start a potential employment relationship by asking me to acquiesce so readily, it doesn't bode well.

Why I won't hire you

1. You want a high paying job but think you're too good to put in some effort in the interview.

Next blog post from the hiring company "why we continue to moan that we can't find devs despite having a shitty coding test".

I am with the blog poster here - fuck you and your coding test.

I lowered myself to do one recently, we walked through it in the next stage interview, where it became apparent the " tech leads" were incompetent muppets, who were intimidated when they saw a better solution then they had as their "approved" answer. Walking through slowly with them why their solution had poorer performance at both low and high volume, I was left with the distinct impression that the vast majority of the incompetent devs who would fail fizzbuzz are somehow doing the hiring.

Scary times!

That's another advantage! If they hadn't given you the code test, maybe you wouldn't have found out you didn't want to work with them as quickly :-)

Nah, 10 minutes in a room doing live coding would have found that out faster than 3 hours of coding and 10 minutes in the room.

"screw you and your interviews"

"I lowered myself to doing an interview recently with some tech leads who were incompetent muppets who were intimidated when they heard better answers than they were expecting to their questions"...

I don't mean to be facetious, but "coding tests can be bad, so I won't do one" isn't a particularly useful stance to take, and I think you'll rule yourself out of a great many good jobs at companies that are very good. I would even say that you might be biasing towards companies where the hiring managers are non-technical (generally not a good sign), or towards companies that lack a solid technical foundation.

I took a few interviews recently and one particular coding test had me really questioning my involvement in the industry - not due to my own competence, but because maybe I really, really do not want to have to go through so much psychological screening and blah before I can get some work done. To be honest, it felt a lot like joining a scary cult ..

Not the company for you, obviously.

What you described sounds like amateur hour on their part, and here's why: if you ask a question which has a single sought after answer, it doesn't give you any data except the binary did they solve it or not. What you want are coding challenges whose answer tells you about that person and how they compare in thinking to others, so there must be a range of valid solutions.

Please don't rant like this on HN. It lowers the signal/noise ratio. Instead, express your view civilly and substantively. There are plenty of examples in this very thread of people doing so. If you don't want to do that, that's fine, but then please don't post here.

We detached this subthread from https://news.ycombinator.com/item?id=11420938 and marked it off-topic.

Reply on news.ycombinator.com