Resumes suck for hiring engineers
blog.alinelerner.com
blog.alinelerner.com
When hiring, you are not really concerned about letting a good candidate go. There will always be others. It is far more dangerous to let a poor candidate in. And poor candidates tend to be poor due to cultural and subject matter issues more than basic technical chops.
I'm not making any hiring decisions at this point - just weeding a huge list of names down to a reasonable number to actually call. So I'm not judging people's skill levels yet -- I'm looking for something in their history that tells me they are bringing more to the table than knowing how to code. I don't even have preconceived notions of what that might be - just something that sparks me to think their experience has something in common with what we do. We'll weed out the weak technical candidates in an initial call, then continue with the people who are both strong technically and a good fit for the work and the culture.
So the whole premise that I am not getting it "right" when judging a person skill based on their resume is missing the point. It isn't that I don't care... but I know reading a resume won't give me an accurate answer, so it isn't even a piece of the decision making criteria yet.
With that attitude, I hope you aren't complaining about how hard it is to find talent.
> It is far more dangerous to let a poor candidate in.
This gets parroted a lot around here, but with no supporting evidence other than highly improbable hypotheticals or black swan events. On the other hand, I know plenty of companies that have had to deal with poor candidates, from a Fortune 500 with 70000+ worldwide employees down to a small 12-person shop, and none have suffered serious consequences from these poor choices. Considering the lack of hard evidence supporting your statement, I believe these companies I am familiar with are the typical case and not "lucky".
- Developer who only worked when being directly monitored. Every feature he worked on was seriously late.
- Sysadmin who had good technical skills but was scared to give anyone bad news until it was so bad the project was in serious danger.
- Developer who stayed stuck at the same junior skill level for four years because he never learned from his mistakes.
- Manager who was too timid to hold his directs accountable, which both decreased team performance and demoralized the team members who cared about their jobs.
I was able to help two of them grow out of these problems, but it took a lot of work. I failed with the other two and fired them, but only after they had caused a lot of damage to productivity and morale.
I saw a junior engineer try to add a patchwork of libraries just because he was not confident in his ability to do something correctly, leading to a lot of maintenance overhead that grew very quickly.
Many of the previous frontend engineers got fired from my current company because they did not produce at all - a direct result is a polluted codebase that needs to be scrapped asap because it has led directly to poor perception of product. It also is extremely difficult to iterate upon.
One company I was at had some very terrible legacy code from full stack engineers. The person who wrote the code was fired as well, but not before leaving damage that persisted for years - some triply JSON.stringified JS object, along with some other extra parentheses, was stored right into the database as a string! This meant every time the data had to be fetched, it had to be parsed by the server to fill certain other data, but the core answer object was still sent as it was due to the difficulty associated with sending a parsed form (the backend was Java), which meant that the frontend also had to parse it! Thankfully we were able to silo the damage to an easy to adapt modular service, but it was the cause of much sadness, especially when it was a core piece of data for our product.
That said, those all fundamentally sound like process problems to me. They would not have been allowed to happen at either of the companies I have worked for or any of the companies I have worked with, at least not to a degree that they would harm products. Sure, we have shit code down in the bowels of our libraries, and some of it was put there by people who didn't know what they were doing. I would not characterize any of it as serious problems, though. Those tend to either not live long or not make it into the codebase in the first place.
I understand there is a tradeoff here, and that "process" is a dirty word to many people. A little process can go a long way, though. It can make problems like those you describe minor or non-existent. To me this seems like a small price to pay for opening up your hiring process and dealing with the poor choices.
On a slight tangent and speaking of costs, I get the impression that many people discount the cost of a false negative. You are trying to hire people for a reason. Your company is incurring costs from not having this person, be it the lost revenue from expanding or the actual costs of working your current engineers harder to meet deadlines. That needs to enter into the equation, along with the costs of false positives and the costs of mitigating same.
I agree with you that the cost of a false negative must be considered. The kind of bad hire I'm thinking of here is a net negative producer that actively costs the company money (or even causes good employees to leave), even after they're gone (if you can manage to get rid of them).
I am starting to feel that if there is some bad code that does some longer term damage, the managers are at fault.
That's why they're called junior and they get paid less. They're meant to be supervised to some extent. If the problem is only caught long after the damage is done then the issues run deeper than the capacities of a single junior engineer. Junior is just a nice way of saying inexperienced. Experienced != good, but inexperienced almost always means some degree of bad.
It is too bad when someone writes less than optimal code, but on the other hand, did the software work? Did it make the company money? If so then it was a success, regardless of how bad the code looks. Often times in business you have to hack together something that works now, and worry about re-writing it later. It's easy with the benefit of hindsight to look back and say how things should've been done differently.
I wish this were true, but the amount of seemingly experienced/employable people I've interviewed in the tech industry who don't understand basic concepts related to their areas of experience/expertise is astounding. As an example I recently interviewed a person to join our operations team managing our VMware stack, who had worked at VMware as an engineer of all places, but could not tell me the difference between paravirtualization and full virtualization or the difference between a virtual machine and a container.
More than anything, it seems many people in the tech industry these days lack understanding of the fundamentals. It's great that you've written a RoR app that makes use of APIs, but if you don't know how HTTP works at least at a high level, how can you troubleshoot that API if you encounter an issue? How can you make intelligent decisions about building out the datastore for your system if you don't understand the CAP Theorem, the trade-offs associated with various data storage engines, and the relation between available memory and available IOPS capability and the TPS capabilities of your datastore.
People seem to have forgotten the mantra "learn to walk before you run" and have gone straight from being dudebros in high school to being full stack engineers or ops engineers without any time spent actually learning how a computer works. Many of the candidates that we end up interviewing have at least 10 years of experience, and still many fail to understand fundamentals.
... but STEM shortage! ;)
Interview like candidates are a dime a dozen, then come whine about not being able to find good candidates
Lots of people say that. Are you sure? Obviously, if one treats people as "resources", and don't let them shine, that's almost true. But I've seen a few times already a single person completely transform the place they work at, sometimes even when not given a chance.
> I'm not making any hiring decisions at this point... So I'm not judging people's skill levels yet -- I'm looking for something in their history that tells me they are bringing more to the table than knowing how to code.
And what that one thing will be that's not a skill? If you (that looks at lots of CVs) don't have any preconceived notion of what you want, how do you expect the people that has this feature will include it on their CV?
Only if you're hesitant to fire poor performers, or your organization or legal environment make it difficult to fire them.
I was recently passed over because my 15 years' of experience with Linux has been with multiple distributions (including RPM-based ones) and my resume didn't have a line that just said "Red Hat" on it.
Them: "Oh, I'm sorry, we can't hire you because you don't have any JavaScript experience..."
Candidate: "What do you think all of this stuff with JS in the name on my resume means?!"
Where a skill is required that I don't have, I don't mention it. Where a skill applies to a version I'm not experienced with, I leave the version number out. When the job spec requires experience of say, Red Hat, and I have experience of multiple distributions, I lie and call it Red Hat. The point of the resume is to land the interview, so I make sure my resume doesn't fail me in achieving that objective.
You absolutely need to do this unless you're in crazy demand. Every generic resume I've sent out has yielded exactly zero phone calls. Every tailor-made resume I've sent out has gotten at least to the in-person interview stage.
Survivor bias? Maybe, but 5 jobs in since college and I'm never sending generic resumes anywhere ever again.
I did find that when I was applying to the bigger SV companies (and Seattle), I wasn't necessarily applying for a specific position, so even if I had wanted to tailor it, it would have been more of a challenge. That's just my experience though. I do believe that if I was going for a very specific position, I would definitely spend the extra time to make sure it was perfect.
I'm sure if I was trying to get a job at Buffer or someplace like that where you are mostly applying to the company, I'd need to rethink my strategy.
(I have no affiliation with Buffer whatsoever).
I don't think Google can give a good match to a Javascript CV with only Angular.js written
Yes, it's stupid. Yes, do it.
I ended up turning down the interview after accepting another offer though.
Yeah, I think you made the right choice turning them down.
If a recruiter or HR rep doesn't know Red Hat from Linux they need to find another field to recruit in.
(a hiring manager is usually the person a new hire directly reports to, not the person doing initial resume screening)
Maybe I used the wrong phrase or maybe it just means different things in different places.
I do not look forward to being on a team with people who were compatible with such a brain-damaged arbitrary system. I've been engaged with too many clients where every day of work is a theater of the absurd.
I've seen resumes in all corners of that plot, so I can't say that I'd throw out the resume straight away, but it does indicate that there is some large gap between what hirers want and what candidates think they want.
Also, a lot of job postings are terribly written, extremely vague or otherwise problematic, so it's not a one-sided problem. It's hard to know if you're meeting the employer's needs when their posting is asking for someone who could engineer a mission to Mars on their lunch break and will work for $50,000 and 1 week of vacation.
I agree. The same could also be said about a companies job posting. If the job posting is "vague or problematic", it also indicates the company has no idea of what skill set they need.
This has many parallels with a client not properly being able to specify his requirements. Usually if a client cannot properly specify his requirements, he also underestimate the amount of work / cost it will require to develop.
The same with a job posting, if they cant properly specify a job posting, an employer also might think the job is easy or he can pay someone peanuts to do it.
If employers wanted to, they could start the ball rolling by simply requesting for more predictive data up-front. Candidates often make specific designs to the company they are applying to. Then companies can filter these candidates out to control the load on expensive interviews.
I'd also say that if a company wants an introspective hiring system, they would naturally pursue a strategy where you identify the kinds of predictors (exclusive or inclusive) you want and request them in resume or some screening test that goes with the resume.
Looks like it goes both ways. Companies will gain better candidates if they stopped using resumes, and engineers will land better jobs if they started creating resumes that are different from what everybody else writes.
"...at the end of the day, the resume is a low-signal document."
"Longer-term, how engineers are filtered fundamentally needs to change."
The conclusion seems to be that resumes are fundamentally flawed.
That is vastly different from reality. When I post job ads, at least 25% of the resumes I get are completely unlike the job requirements, and another 50% superficially match but are obviously unqualified. The remaining 25% look like the resumes I saw in the survey.
When I get resumes from recruiters, I generally don't get any of the "didn't even read the ad" resumes, but I still get about 1/3 obviously unqualified candidates.
So overall, a resume filter lets me skip hundreds of useless phone screens or face-to-face interviews. I may lose a few good candidates in that process, but not many.
I'm really not sure why this author is trying to use resumes to decide whether to interview these candidates. Once you have them filtered to this level, a quick phone screen gives you much better information than a resume, and doesn't take too much time.
Resumes can work for weeding out bad fits, or people doing things clearly out of the scope of interest. They are not, and cannot, be a tool for choosing the best. (Unless success is so spectacular, that you know the person by name anyway.)
It's HR's delusion that previous -perceived- presented achievements written on one's resume can serve well for predicting future achievements in our company. So, don't overoptimize it an do technical interviews with people actually working on a given technical problem.
Maybe it is just me, but I don't do puzzles. I solve big problems, and I have a track record of being very successful at it. I juggle people, and tools, and objectives, and timelines, and get things done.
I like projects, and I am not afraid of hard work. If I like the objective of a project, I will wade through human vomit to get it done.
Brain teasers are none of that for me. They are some kind of parlor-trick brain-test. I am not necessarily bad at them... but I am just a bit better than okay. I do not want to have to practice them just to get past an interview.
The presentation is quite manageable. The brain teasers I said no thank you to, because I would probably just make a balls of it and feel bad for two days.
That is the point. The experiment was to determine who various people would call for an interview, not hire. Nearly every resume was worth a phone screen, yet on average half of them were rejected for that stage.
http://blog.sfgate.com/gettowork/2014/10/31/staffup-offers-a...
We recognized that resumes were poor indicators of performance. In fact, we believe that the only thing that correlates with performance -- is performance. A study that Google released last year supports that.
The solution we came up with was to invite people to work on projects for a weekend at no cost other than their time. They chose what they wanted to work on together.
In a sense, we were batch processing applicants while gathering much richer information than resumes provide. Strong candidates we didn't hire, we shared in the Sequoia network.
We ended up making eight interview offers (several candidates decided to work elsewhere), and we hired one engineer.
The whole event was inspired by this essay: http://brookeallen.com/pages/archives/1234
I see more and more companies doing this small projects type of initial screening. I like it. But time is actually an expensive cost. Suppose your candidate makes $130,000 per year, which isn't even a high salary in bay area. Then if he/she spends 15 hours on this project, then we are talking about 1000 bucks. Now that is quite a cost, isn't it?
Honestly compared to a white board programming puzzle, such projects always make more sense to me. But most companies will still do white-boards during on-site after the initial project. And although I like most of those projects, even in my case, if I happen to be busy, then I do feel like they are annoying. I'd rather spend my time doing what I was busy doing, or learning new stuff, or working on my pet projects, rather than working on a project that may or may not make sense to me, just to show to other people that I can code, and I can write clean code. Let alone I happen to know some friends who just hate those projects and will never do them unless it's from their dream company. Plus, there are people who are better communicator when they talk face-to-face. Then it's really hard to say if such project is really better than white-boarding for that type of candidates. But it's still way better than those 30-minutes online coding over a phone interview. That's the one I hate most.. Even Skype interview is better than phone interview.
Instead of presenting the "screening project", they present their open source project and its source code.
As in working for you for free, and on a WE to boot?
For a student or someone who is unemployed, that might work.
For someone with a job and a family life, I don't think so. Especially if that person can command a $120k+ salary.
We invited people to come up with ways to improve the job market, which is clearly broken. Some chose to do that, while others had their own apps already in mind.
One group made contactr.io, an extension for Google Chrome that shows you the email addresses of the founders of the company website you're visiting.
Another group is working on a project called livelyhood, which will be a sort of LinkedIn 2.0 where candidates can share their work easily.
The participants worked for themselves, helped each other, and walked away with a larger network of people who recognized their qualities.
This weekend was specifically designed for people whose resumes would not automatically "command" a $120K salary. They were the ones languishing in the slush pile.
We held the event on a weekend so that people who already had jobs, albeit jobs they were unhappy with, could join us.
If you like your weekends more than good work, then it definitely would not have been the right fit.
Seriously????
I'm sure the OP means well, and it may have been a success for them, but I don't know anyone in my circle who would participate in something like this, unless they were utterly desperate for a job. I thought there was a serious talent crunch? Who is courting whom?
I wonder what sort of engineers self-select in a process like this?
Unemployed people have lots of time. Understand that and you'll understand why a weekend like this makes sense.
One reason they're unemployed is that they don't know how to demonstrate their value. A lot of engineers are actually poor at communicating, and not that great at resumes.
This weekend was for them.
15 weekends of hard work just to change job? Yes, very efficient.
The arrogance of your approach makes my blood boil.
We had several positions to fill, and we helped good candidates whom we couldn't hire get in touch with other employers. The chances of an interview were much higher than 1 in 15. In fact, we made eight interview offers.
The weekend was totally voluntary, not cheap for us, and everyone knew exactly what they were getting into. It was one weekend, not 15 weekends.
If you think weekends like that are inefficient, consider the job market as it operates currently. If you can think of a more efficient way for it to work, I would love to hear it.
The bit where you call me stupid...
Compared to the other very unsuccessful interview, where the MD just listened to me talk for a few minutes and said "you're not what we're looking for" (no programmer in the interview), getting engineers to ask me end-of-interview questions with a smile on their face was really fun (whether or not I get the jobs in the end.)
The OP says that engineers were quite good if the CV contained enough detail about what the candidate had been doing, the substance of the projects. In both cases, the little question was where I had included a bullet thinking "noone in HR will know what this means, but I think it's cool" were the ones that got the smile question.
For the record, all my CVs are skills-based (that is, a set of bullets around "programming", "teamwork", "communication" etc) followed by a brief chronology of work and education, rather than having to list every position (student volunteering included) in order to say what I did there that was relevant. Customising which bullets (and even sections) you include on a per-application basis is extremely important, of course, so that the HR person can see "skill that we asked for: 5 bullets" right there on the first page.
I for one welcome phone interaction and tests, though not all tests perfect and found many flawed tests and errors in tests in the past. Some companies appreciate when you tell them of mistakes and others will hold it against you, though again, found HR departments if you point out a mistake will not take it onboard and take it as criticism and in a negative way and hold that against you.
So for me, I do feel HR departments play too large a part in recruitment in IT than their skillsets entail.
Not saying a golden solution, but I do feel that many recruitment processes have gone down hill with HR wedging themselves into the process more and more and filtering out candidates that the manager doing the recruiting, never even knows about.
Though this is all based upon my experience and I'm sure others had different views. But for me, having been turned down for a job for being overqualified and case of you actually knowing the manager and he never even got to see your CV, well, sadly I know it is very true and happens.
Yes, but you can't call everybody that sent you a CV. RH filtering sucks, but is needed usually. Of course there are false positives and negatives
Yes, RH might miss a J2EE expert for a position requiring Java experience, but these days companies search from big databases (Linkedin/Indeed) so this type of grepping is usually needed.
Because most of us wouldn't want to work for a company with such a broken hiring process, it's a big red flag for a lot of other things being wrong in they way they treat their tech people.
And those who do want to get in should be smart enough to play the game and cater their resumes to those HR practices. If you can't or won't, you probably wouldn't happy or successful working for such a company.
As far as I'm concerned, any recruitment process is essentially self-selecting: it will attract and pass the candidates that are a good fit for the company. If a company finds it can't get the right candidates, they should take a step back and look at their entire company culture.
Broken HR processes are a symptom, not the root cause. And it's a filter that works both ways, preventing naive candidates form inadvertently getting hired for jobs that will make them miserable.
Accountant exams and lawyers as the best examples have not changed. So for HR to see filter is not only easy but obvious process in such stable measures. In IT those change so much that it is mostly recruitment agency input that has those factored into requirements and as for agencies. Well some good at filtering and know the trades and others mostly work upon (1) get CV's (2) get companies (3) push CV's and effort done by wine and dine approach with HR.
So not only is it the HR department but the agency factor and in the web based direction the added level of filtering does get down to keywords still and little if not mostly none at all human sanity by somebody who knows the role and can see the signs of somebody who knows the skills.
Then keywords do not bestow what level of competence and that varies greatly as we all know.
This all said sometimes the gains outway the broken HR if you get an offer and not every time do you have the luxury of saying no.
Though as a personal anecdote I once had a job interview with a company and got there on time, left waiting and eventually HR bod came down and said fill this in (form which was all details upon my CV), so I ask can I just attach my CV you already have and told no. So I had to fill out in pen on a form that you just know somebody will have to type into a system, details already upon my CV. Did that, still waiting for HR bod to return. They return then have interview with HR. Was very belittling with much attitude of thats a lot for money for somebody your age and in short really off putting attitude. I then saw the boss of the IT department, was interview with him and IT contractor, not only aced the interview and blew them away but taught them things as well like undocumented debug modes in software they used (dataease so talking 90's here). Given the contractor was a `specialist` in this product he was mind blown. Also the small bonus skills I totally aced, in short a perfect interview at every level.
Not only was I offered the job before I got home from the interview but they had offered 20% more than the agency had put me forward for and that HR person had been so much attitude towards me (I was early 20's). It just totally put me off, and turned it down, just due to that HR experience. Though have many other HR anecdotes, some good and some bad, as we all do and shall not mention the posture nazi's for want of a better term.
It's a good thing for hiring process, you could judge a candidat more easely if you could read some of his code or thoughts. You could have more interesting job offers from people that actually understand what you do. The problem is that you have to spend lot of time in improving these things. Some are easy because it's natural to do some work on a open source project that you're interested in, the personnal branding thing it's just a bonus, but some other need works...
I believe that in all automatic job applying process that I tried recently there were fields for linkedin, github, personnal blogs, websites etc.
Her theory - people suck at filtering because the tool they use is poor.
> it may not be a matter of being good or bad at judging resumes but rather a matter of the task itself being flawed — at the end of the day, the resume is a low-signal document.
Resumes suck for hiring all people. But that's not the purpose of a resume -- it's a low-signal filter to weed out the initial batch of candidates. If you treat hiring like a sales funnel (and you should), then a resume is a great way to eliminate unnecessary time for your team to assess someone's cultural fit. Expecting the resume to be high signal filter is pretty naive.
Where's the evidence that it is great at this? Certainly, its a basis for narrowing the set of candidates that get further review, but is it a good basis?
I was slightly perturbed by the original author as she offered no real solution.
Where's the evidence that resumes are better than random selection among applicants into the next round, much less enough better to be a net positive given the resources that go into reviewing them?
Well, yeah. If you don't have any better metric, the best result comes from filtering out everyone who just blatantly submitted a resume for the wrong job entirely, and then selecting from the remaining pool at random.
A company is made up of people, hiring is to a company as mitosis is to the human body, it's fundamental. The better you can be at hiring the better your company will be. More talented, more capable employees translates to better execution on projects, better ideas, better products, more revenue, a higher profile, and a greater pull for more talent out in the industry. If you treat it like boiler room telemarketing grabass then you'll end up with a company to match those ideals.
Does anyone else think this is a huge caveat getting thrown into a blender of numbers? Why does this person's opinion even matter? Why should I take this at face value? I'm not trying to say resumes are a good indicator of anything, but neither is this "study".
> To try to understand whether people really were this bad at the task or whether perhaps the task itself was flawed, I ran some more stats. One thing I wanted to understand, in particular, was whether inter-rater agreement was high. In other words, when rating resumes, were participants disagreeing with each other more often than you’d expect to happen by chance? If so, then even if my criteria for whether each resume belonged to a strong candidate wasn’t perfect, the results would still be compelling
The result of the Fleiss' kappa test subsequently run was negative, i.e. people didn't agree with each other either. So maybe the author's judgement was wrong, but that doesn't affect the conclusions.
But what I or my company consider a quality candidate may differ from the author.
On another note, I've found that for recent grads, GPA, school, and major (all items on the resume) are excellent predictors of interview performance.
And honestly, my primary metric is if the word "rails" appears anywhere (It's a ruby on rails job). I might even let that go if I saw a cover letter explaining why they liked rails (but maybe don't have professional experience in it).
Honestly this is a pretty low bar. And the way I've been doing it, the phone screen that follows is pretty softball. But after that, I give out the coding exercise...
If you complete it, you get an in person interview for the most part. But all the rest of it - looking at resumes, the phone screen - sort of seems like a waist of time. I don't care how big a game you can talk, if you can't do the coding exercise (which, btw, is the most common outcome - they beg a couple extensions and then give up), then... oh well.
There are many I've interviewed that padded their resumes and said they are experienced in something they really aren't, and there are some who dont have much experience but are smart and quick learners.
Without a resume, what else can we go with when it comes to hiring? In my career, the best hires have always been from two sources:
- someone i've worked with and trust before
- someone who was recommended by someone i worked with and trust before
The company I work at building a way to capture those recommendations and that concept of trust through peer-to-peer reviews: https://www.roikoi.com/
The site asks you to play a anonymously game comparing your most relevant coworkers to one another and a users' score is generated by their aggregate results from their coworkers playing the game. It essentially crowdsources gauging those qualities that can't be gathered from a resume.
All of the leaderboards for the companies that I worked for are all top notch folks I would hire in a heartbeat, so so far our output has been fantastic.
I wrote exactly these two tags "fast thinker" / "quick learner" in my CV / cover letter / LinkedIn profile and I removed them a few months later because from my experience it looks like they drive recruiters away or scare them.
You can easily determine if some one is a BS guy in a technical interview anyway, so I don't think, BSers are very frequent in IT and much less so in programming.
And I'm not sure I wouldn't get negative vibes from someone who felt they had to position themselves by de-positioning the people they worked with.
>You can easily determine if some one is a BS guy in a technical interview anyway, so I don't think, BSers are very frequent in IT and much less so in programming.
Of course they're frequent. They are everywhere.
I've got no magic bullet. Hiring people you know is good, and I strongly recommend informal face-to-face interviews. Two of the best interviews I've ever had (one where I was the interviewer, the other where I was the interviewee) were mostly about language design, which turns out to be a really revealing topic for discovering attitudes about process, standards, and software design generally.
But how to screen people in the pipeline so you can avoid interviewing everyone who applies? No idea.
To complicate the situation, most of phone screenings I did were made by young recruiters that don't understand anything of what they are asking about, making all this thing silly and awkward. A skype interview is much more relevant but probably scary for the (non technical) recruiter.
By the other hand if I can get a face to face interview, normally I'm in.
There's been a lot of data about how even people who think they are experts at finding good candidates are really no better than someone flipping a coin.
How then, if the line between "strong/less strong" candidates are so blurred, does Aline draw the line between "strong/less strong" herself? Isn't the way she splits the candidates' resumes also part of the exact thing she's trying to measure?
> "...then even if my criteria for whether each resume belonged to a strong candidate wasn’t perfect, the results would still be compelling"
> if the line between "strong/less strong" candidates are so blurred
It is hard to tell prospectively whether someone will be a good programmer. It is not too difficult to tell retrospectively. (Any research that tries to measure competence/success indicators must be using something as its dependent variable, right?) The line you quote is specifically about the fact that resumes could be meaningful even if her competence assignments were not. Resumes seem to fail at that test as well.
The biggest threat to validity would probably be that different people will be good/bad developers in different environments, and that this effect is very strong. Then each reviewer could be good at selecting for success in their own company, but they would only agree with each or with aline to the extent that their environment was similar to that of other companies or the ones that aline placed those engineers at. I am pretty comfortable rejecting this theory out of hand myself.
It's also a way (in addition to the cover letter) to quickly judge their communication skills.
I wouldn't hire someone without 1) a cover letter 2) a resume 3) a phone screen 4) review of their projects (ie github) 5) a team interview 6) references. Granted, I don't have to do it at scale.
It's time consuming, but not as much as a bad hire.
Should there be something other than a resume like a timed coding test or something (where the applicant needs to take half an hour and write a program to do X in the language that they will need to work in, probably something really simple)?
You might not mind that filter (hiring cheaper but talented young people), but it's surely a bias.
I come from academia. I know what its like to work toward a goal for the sheer pleasure the challenge brings without expecting (much) compensation. But that's academia. This is (typically) private, for-profit industry. There is no good reason to not demand fair compensation and respectful treatment that includes not being given unpaid monkey-work to "show off" for some ass with a superiority complex.
http://www.amazon.com/Dont-Send-Resume-Other-Contrarian/dp/0...
It's a good collection of advanced marketing techniques for job searchers.
Arrogance, disrespect and plain unpreparedness.
They're the first step in the funnel, if you can't get through them your resume usually needs fixing.
And of course the interviewing flaws that are discusses several times on HN play a part as well.
Apart from that one, the consistently well-rated CVs are all from the strong side.
So it could be argued that CVs work pretty well for recruiters - hiring from the best CVs makes it unlikely you'll hire a weak candidate.
It seems there are several audiences for a CV (recruiter, hiring manager, supporting interviewer) and a fundamental error that people make is writing a CV for more or less than these three types of people.
The fundamental truth is that no-one wants to read a CV. They're at best an advert, but people tend to overstuff them or fill them with things irrelevant to the position they're applying for, so they get ditched by people who have tons of them to read through.
I have a free email course about career hacking for anyone that subscribes to my book's mailing list. It's focused on people who want to become penetration testers but the career material is applicable to anyone wanting to get into a discipline that they're not already in.
[1] - https://www.rawhex.com/
It's also worth noting that several of the criteria people used to reject candidates in the article aren't good predictors. For example, lack of pedigree doesn't tell you much. People from a variety of backgrounds can be good or bad as coders. A good or bad pedigree can be a useful indicator.
Things like communication skills, priorities, attention to detail and motivation are strongly reflected by someones resume.
The only people who can afford to not put serious effort into their resumes are people that have an online (Github etc) or offline (network) footprint that speaks for itself. But those also tend to be the people that don't get recruited via the formal application process.
It wouldn't shock me if it also gets used as code for things like "hasn't worked at a startup before" or, frankly, "too old."
The two main points in the blog stated that an "enterprisey" developer may not be a good fit in a startup (change in development pace, responsibilities, etc.) and that they may use languages unpopular in a startup environment (Java and .Net are called out in the post).
I'm interested to see if there are other posts that echo this sentiment, aside from the OP, because now I have a fear of being "too enterprisey."
https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
https://news.ycombinator.com/item?id=8338859 https://news.ycombinator.com/item?id=8532886
Seems like SF is on top of it, Milwaukee on the other hand, only cares about how many years of experience you have. Nothing else.
Testing like SATs are similar. A low score is probably a much better predictor of (lack of) success at an elite school than a high score is a predictor of success.