People are still bad at gauging their own interview performance
blog.interviewing.io
blog.interviewing.io
This is the key takeaway for me. This is IMO and experience probably based on a "sour grapes" type defense mechanism, where candidates will internally try to talk themselves out of wanting a job that they felt they didn't get. They walk out of the interview feeling they didn't perform, and instead of regretting the lost opportunity they start to focus on even the most minute "negatives" about the job/company.
The more time you give a candidate to stew about those reasons they don't want the job, the more time they'll have to find reasons that may not even exist.
Positive feedback should be given almost instantly. Some companies and candidates are reluctant to give positive feedback quickly because they feel it may hurt their negotiation leverage.
I don't have data on this, but anecdotally (over almost 20 years in recruiting) I've seen this countless times while collecting post-interview feedback from candidates. The ones that feel they did poorly are likely to mention a negative about the interviewer or company, which becomes somewhat awkward when we come back with a job offer.
"Would I want to work for a company where the employees pass their given process?"
And rightfully so.
The term these days is "candidate experience" for the overall interview process, which can include everything from a company's handling of logistics and time spent in a waiting room to the types of questions being asked.
That said, what I'm referring to in my comment isn't necessarily related to the interview process at all. A candidate can be asked questions that he/she felt were relevant and entirely fair for the job at hand, yet if he/she performed poorly there is still a tendency to try and find flaws in the overall opportunity.
My reference wasn't to candidates performing poorly and thus giving a bad review to the interview process. Rather, the candidate might perform poorly and say that the vacation time was inadequate or the commute would have been too long.
At this point, isn't it generally accepted that all tech interviewing sucks, and companies would do better throwing darts at resumes?
If all interviewing is bad, and all companies are not bad, there's no benefit to judging a company based on that process.
Were they on time to the interview? Did it seem like a good culture fit for me? Do they seem positive about working there? Was the interview full of poor communication? or confused looks?
Everyone who works at a company, got there by passing through the] interview filter. A bad interview process has everything to do with the company, particularly in organizations that have grown a lot in the recent past.
At the time I was interviewing with another company who seemed interested in me, but expressed a deep suspicion of new hires after having been burned in the past, and told me outright that they were skeptical of me, and that I could very well turn out to be lazy and unproductive. After three interviews in three weeks, they still hadn't made any offer. So when another company comes along unsolicited and asks me to join them because they know I'm good at what I do, I took their offer. When a company demonstrates that they value you, you're going want to work for them instead of the one that doesn't.
There are a few people at my current company that were hired with an interview based on the company needing skills that weren't available from referrals from employees, but that's uncommon. I don't even know if our company has any sort of standard interview process.
I think it's a bit more nuanced than that. Most companies seem to be interesting in improving their recruitment processes - but certain fundamental systems and assumptions are treated as inviolate and sacred.
Here is a quick but incomplete list of the ones that come to mind:
- Open-ended unstructured interviews are the optimal way to evaluate candidates.
- Whiteboard coding is an acceptable, reliable, and reasonably accurate way to gauge coding ability.
- Algorithms and data structure knowledge predicts overall software engineering capability.
- Culture fit (in however nebulous ways) is a stronger predictor of future performance than standardized evaluation.
The list goes on.
The tech industry overall seems very interested in improving recruitment and hiring, so long as these fundamental tenets are never violated. Suffice it to say, this stands in contrast to tech's (in)famous claims of thinking outside the box, breaking the rules, and holding nothing sacred to disruption.
I've personally seen companies try to improve candidate sourcing, scoring systems, interview scheduling, etc etc, but practically never any of the above. Companies try really, really hard to optimize within an extremely narrow band of parameters.
What they don't really want, is to take on people who are noticeably better than (or different in any way from) the people already working there.
1. Pushback on work sample tests from the recruiting companies.
2. Upper management wondering why it's so hard to find candidates that can pass our work sample tests.
3. Suggestions that maybe we should be using an online coding test. (multiple choice questions of poor quality)
It can be really difficult for companies that don't have a lot of in-house experience here to figure out how to hire good developers. Much of what is known in Silicon Valley style companies isn't really known by HR in a smaller company in the midwest. They don't typically read HN or follow the tech blogs. And unless they luck into hiring a vocal Senior Developer like me to specifically guide their recruiting and interviewing process they won't even realize how badly their hiring process is.
Except I've had good interview experiences. So, no, there are better and worse kinds of interviews. Just because it is an overwhelmingly neglected aspect of the industry is neither here nor there.
Or perhaps by actually reading resumes instead of keyword-filtering them. Really, it's not that hard.
"Throwing darts at resumes", OTOH, is exactly what many companies seem to be doing right now. Specifically, the practice of setting up a "chat" with an internal recruiter (who inevitably tells you that they think you'll be a "wonderful fit") before passing the resume over to the tech side (which is when you find out whether they really think you're a potential fit or not) comes to mind.
This one is a two-way street. I launched a resume writing side-business a couple years ago (resumeraiders.com) because I saw how poor many developer resumes were and how they were getting ripped off by resume writers who had no experience in hiring or in tech. Most were just failed writers.
If you expect a recruiter to read a poorly-written 5 page resume when the yes/no interview decision could have easily been made based on a summary and a few decent bullet points, that's not on the recruiter. If you think the 5 page resume is some kind of anomaly, ask the recruiters at your company how common they are.
Her resume was 1 page. It's the first time I've ever seen a 1 page resume in my life except for someone out of highschool with no history or experience.
Mine is 4 pages. I've worked really hard to cut it down to that while including relevant things and nice formatting/readability/white space. Compared to some of the 10 to 15 page resumes we get here for people who have done so little, it's positively succinct.
We had a chat, and we just had to agree that it was a cultural difference.
But still, I admit, if anyone asked me here, I'd say "you can't have a one page resume, unless you are literally applying for McDonald's and have done nothing with your life, this isn't twitter"
For government jobs it's even worse, there you basically do have to write essays against the "selection criteria" (don't get me started on what it actually achieves, again, I'm not trying to advocate).
The general rationale is often on several grounds:
-to create a "merit" based document trail - to create a basic barrier to entry that has to be crossed for all but the most basic positions. Think of it like a spam filter/ensuring candidates who want THAT job as opposed to just A job. The very thing you don't want is to have hundreds of 1 page dot point resumes coming in. How would you possibly find the time to go through them? You want to stop people applying just by sending in a generic summary resume and clicking a button. You want people who are willing to put a bit of work in to apply specifically to the job you're advertising.
It is not uncommon for me, for instance, to write a cover letter for each job, and craft a specific version of my resume for each position, emphasising and exposing those parts of my experience most applicable to that particular position.
You can only really do one or two of these well per week, but that's kinda the point.
Almost anyone can fit their entire experience onto a single page. You don't need to write an essay for each position—just highlight a few bullet points.
Oh sure! I can cut it down to 1 word if I want: awesome :p
One of my points is, I have literally not seen a 1 page resume before outside of fast food/janitorial work. Before chatting with her, I would have advised her to fix that up, because no one will take her seriously with a dot point resume that looks like it was put together to hit some automated text search and no natural human reading. I admit after some discussion, I fell back to "wow...they're not even pretending they're nothing but a keyword search meat grinder", because I do respect her so I thought she was telling the truth. And now I've got additional corroboration from HN.
I think there are arguments for longer/shorter resumes. In my, and my jobs/culture defence, I need to see you can write in plain English, have some sense of aesthetics, prioritisation, presentation. By that age I expect you've worked in several jobs (about which your summary might be my first impression/knowledge) and dot points aren't going to cut it, because your job titles, product names and keywords are meaningless noise to me: I joke my grandmother is an entrepreneur/CEO and has worked with everything. Your dot point resume looks like an indian from mechanical Turk put it together by amalgamation of keywords from job ads and pop culture. I want personal details, but summaries, on your projects, products, papers, in plain communicable error free English. 1 page screams kid/unprofessional to me. 2 - 4 pages unless you're 50 - 60 and have written 100 books, done something truly remarkable.
Now I'm not saying its right, I'm just saying that's my world.
> Your dot point resume looks like an indian from mechanical Turk put it together by amalgamation of keywords from job ads and pop culture.
> 1 page screams kid/unprofessional to me.
Your 4-page resume is unprofessional, overlong, self-centered, arrogant, and naive. You apparently are so bad at English that you can't even write concise summaries. A long resume screams that you have an overinflated ego and have no sense of professional decency.
A short resume doesn't imply a lack of aesthetics or English. I certainly paid attention to the design of mine. [0]
If you can think of a nice way to convey the same sentiment/message, but without the offence, I'm all ears and would happily edit the post.
Again, let me stress, I think there are arguments for long and short. I do not advocate for right and wrong, merely try to convey the reality I'm in, given that it's so different apparently from SF.
Realise also, in good faith, how offensive the "1 page resume" sounds to my ears. There is not any hyperbole present when I say I haven't ever seen one. Indeed, the only reason I believe it exists is because otherwise intelligent people I respect have told me it does, and now hacker news backs that up.
Really though, is it so hard to believe that people think that summarising yourself down to a page (as opposed to three or four...lol) is also an offensive idea?
Are we really so departed from one another that we believe that we can't just agree that the way our two respective cultures do things is offensive, especially given the fact that neither of them appear to have much success with outcomes regardless of which culture is in play?
But...
You do realise what "mechanical Turk" is...as in...the Amazon service called "mechanical Turk" based upon the chess playing automaton, and that it's basic premise is often westerners paying people from poorer countries, say...India, to spit out quick small pieces of work for them.
There is no offense intended in these statements :/
It's offensive to imply that people with 1-page resumes just had it generated via mechanical turk. I've spent hours perfecting my resume to have a good layout and clear copy so it could fit on 1 page.
You really need to adjust your value system. Writing a short resume is harder than writing a longer one. In fact, I have a 4-page "master" resume which I pull from to build my 1-page version.
> There is no offense intended in these statements :/
Of course there is. You don't call skilled professionals "kids" unless you're trying to be offensive. Don't play dumb.
One of the important lessons we learn in this life is that we're responsible for the offense we cause (or more specifically: the offense implicit in what we say), not the offense we intend.
The resume doesn't need to include everything you did. It's a highlight reel, not a bio. You don't need to list every responsibility or even every accomplishment.
If I forced you to tell me your background in one page, you'd be able to do it if you have any sense as to what is most impressive, what is important, and what isn't. The reason your resume is four pages is that it's acceptable where you are.
I once reduced a 26 page resume to a couple pages for a non-US client. It's usually just triage and deleting, some sentence "refactoring", and formatting. Not too tough, as long as my client doesn't have too much emotional attachment to every detail from their career.
That reminds me of another technique - that is you select roughly half of the resumes from the stack and bin it. Nobody wants to work with unlucky people ;)
"Eh, I don't have time for this. Let's get the junior people to filter resumes and do phone screens. Especially when trying to hire senior people... Too many resumes? Just make them do a 3-hour HackerRank session first. It's what all the other places are doing..."
The rest of the company didn't care enough to fix a bad interview process. That has plenty to do with the rest of the company.
>isn't it generally accepted that all tech interviewing sucks,
No. The interview process I went through was phenomenal. And my job is very pleasant and rewarding.
One of the things that really struck me at the time was that traditional interview processes are built around a notion of worker surplus and job scarcity. They make very little sense when the opposite condition applies.
Even little things like contacting all applicants within 24 hours really wowed people.
The market is likely inefficient for the same reason that the advertising industry is inefficient - difficulty to show than any given change independently improves outcomes.
In tech, though, you hear endless complaints about how hard it is to hire, so I didn't hesitate to optimize the process for interviewee happiness. I expect it would make people more likely to apply, more likely to stay through the process, more likely to accept an offer, and more likely to refer friends.
Regardless, there's enormous waste to be removed from the process for anybody who's willing to put a little thought and work in.
This is true for a lot of companies - but there's only a surplus of labor if the applicants are qualified.
For a company to be able to be frivolous with their process, they'd need to have more qualified applicants than jobs, and I think even for many name-brand BigCorps, this isn't the case.
I'm convinced that's the only reason why processes at places like google aren't considered completely broken.
Some of that filtering, whether intentional or not, is how rigorous the interview process is. They can afford to be that rigorous though because the fact is that even if they miss out on one really good candidate there are 10 or more replacements in the pipeline.
The same does not apply for smaller companies with less engineering centric workplaces. If they try to do it Google's way they'll be shooting themselves in the foot.
I honestly find it almost disrespectful when you feel some company is putting you through a pipeline. They set up an interview day, line everyone up, grind through them one by one.
The other thing is the technical interview which is unreflective of the job itself. In one recently, I was told before going in I had to code on a whiteboard. I ask, "Can I have I use a text editor, not an IDE or anything, just a text editor?" I get told, no (for nonsensical reasons I won't list here). That sort of arbitrariness and inflexibility turn me off, not to mention the standoffishness of having people stare over your shoulder and judge you rather than work as collaborators.
So I walked away from the interview. I'm happy where I am and you're gonna have to woo me, just a little bit, to get me to care.
Sometimes I wish we were unionized so we could set some standards (for many other things, but this as well).
One of them mentioned feeling less pressure to have perfect syntax and everything at a whiteboard than in an editor. In retrospect, wish I'd asked more of them about their preference, although at the same time, it wasn't something I judged them on so I wouldn't want them to think I was nitpicking their choice.
I'd probably choose a whiteboard too, honestly, for spatial reasons (able to write up separate things off to the side, easily annotate stuff in a different color to step through the code, etc).
My theory was that in an interview I wanted to see people at their best. I wasn't hiring them to code on whiteboards or learn how to use new text editors; I was hiring them to spend a lot of time coding in an environment they liked.
More details here: http://williampietri.com/writing/2015/slightly-less-awful-hi...
It's not just going to be 'sour grapes' but also going to be preparing themselves for the rejection they now believe is coming. They're making it so they're not surprised when it shows up or when they don't hear back.
They're also making themselves more open to other offers instead of focusing on this one.
Wait. They think they didn't do well, got an offer, but turned it down because they think they didn't do well on the interview?
The blog post just refers to interviewees saying "I wouldn't want to work with the interviewer" when they mistakenly feel that they did poorly in the interview.
Anecdotally from the real world, when I (headhunter) get feedback from a candidate shortly/immediately after an interview, if they feel they did poorly they tend to add that there were things they didn't like about the opportunity (a person, the office, something). People that feel they did well don't usually go negative as easily.
But I also go back to the employer and get their feedback ASAP, and when that comes back positive I immediately pass that message to the candidate. Assuming the employer's feedback is somewhat prompt, the candidate doesn't get too much opportunity to go negative, and once they "feel wanted" by the employer they usually trend back positive. Offers come shortly thereafter.
Yup. Happened to me: I felt like I did poorly in one of the interviews and had to ask myself whether I'd be happy in the job if the day-to-day was like the interview.
It probably wouldn't have been, but I had another offer on the table and the negative experience swayed me in the direction of the other offer.
If I think I did poorly on an interview, I will probably stop following up with the company and will more aggressively pursue other options. In particular, I might accept another offer prematurely because I assume one isn't forthcoming from the company.
As the post mentions, this relates heavily to timing. It's the long delays which many companies introduce that force candidates to make assumptions/prune early, especially since some companies will never even tell you you were rejected. Speeding up candidate responses is one of the easiest ways to dramatically improve a hiring process.
For candidates who don't do well? Thank them for the time and move on.
This approach should also tend to reduce counteroffer acceptances or accept/renege scenarios.
In other words, I don't think that's a good solution at all.
Of course lack of a timely response also reflects badly on the company, so that could also be a very valid reason for people to lose interest.
Without knowing what the interviewer's bar is for any particular star level, I don't see how the interviewees could do any better. Rounding to 1-star increments amplifies relatively small changes -- if the interviewer rounds up 3.6 to 4 and you round 3.4 down to 3.
It is pretty clear from one of the plots that interviewers and interviewees grade on a different curve - the mean interviewee scores themselves lower than the mean interviewer does. But once you correct for this, it seems that people are actually pretty good judges of how well they did. I'm honestly surprised by how low the R-squared was, I wonder if they included an intercept?
For the purpose of this article, interviewer ratings are considered the absolute truth while interviewee ratings are not, which is obviously not always the case in reality.
Maybe this is because the interviewee feels partly abused, as if communication isn't a two-way street. I think interviews would be less terrible if interviewers actually told you at the end that you did above/aswell as/below expectations, and/or why instead of just waiting 1-2 weeks before getting back to you and then telling you that you are/aren't moving on to the next step. A huge part of the interview process produces emotional ups/downs that no one wants to deal with and which ostracize potential employees, but the interviewers don't really do much to help with this.
I also think that having a 1-4 range is not a good guide for gathering sweeping datasets. Adding any amount of numbers will help to show actually how much. Or actually using human language to define them would be better. 1) Terrible/Unhirable. 2) Unhirable/Not Passing 3) Passing 4) Exemplar would actually be beneficial right now arbitrary numbering doesn't help to give people much context as to what they should choose, and the numbers are so close that you're going to get outliers where maybe some people think that a 2 is passable but not great, and interviewer thinks 3 is passable but not great, or vice-versa.
This happens quite often in phone interviews. My clients have regularly told candidates "We'll be inviting you in" at the end of a phone interview.
The issue with this for on-site interviews is that they tend to be with multiple individuals/groups, and getting those groups together to discuss the candidate's performance while the candidate is still on-site is a mess logistically. It would be ideal, but usually there will need to be some discussions among the panels.
The way we've handled that (startup, ~30 devs) is that the VP of Engineering goes last, and will go around getting quick reactions from everyone on the loop. If everyone says "I'm all for hiring", the VP will go ahead and let the candidate know we'll be making an offer. If anyone says "I'd like to discuss", then we don't give feedback until after getting together to discuss the candidate.
There are lots of positives in providing that immediate positive feedback - showing the ability to make a quick decision, organization, and not least making the candidate feel wanted right away.
Dunning-Kruger effect is attributed to "a metacognitive inability of the unskilled to recognize their ineptitude." [1] Seems like a harsh accusation for being off by a single star.
And the author makes a generalization that the imposter effect is "better" than Dunning-Kruger effect. I don't clearly see why this would be the case for the interviewee.
Pulling some piece of psychology jargon out of thin air doesn't make it immediately applicable or accurate. Presumably, the terms get used so frequently for the "look how smart I sound" effect, which serves only to piss me off.
I would categorize these two groups as "overconfident" and "under-confident", at most. They're not suffering from some genuine long term psychological condition just because you think it sounds cool to use the terms.
/rant
In my experience, the problem is that interviewers have no idea how to correctly value a candidate's performance. Maybe the candidates are closer to being well-calibrated, but their self-assessments don't match up with the interviewers' because the interviewers don't know how to gauge what they are looking for?
Making the assumption that an interviewer knows how to measure the response of a candidate, even in cases of extremely quantitative questions with well-defined answers, is highly suspect to me. I think virtually no one knows how to do that effectively.
Strange experience.
We certainly can speculate whether whose opinions are more correct or valid, but if we just objectively consider that doing perfectly means you get hired, then what the interviewer thinks matters and what the interviewee thinks doesn't.
I do think you raise an interesting question though. I do wonder whether, in a scenario where many interviewers saw the same interview performance, how varied their scores would be.
After all, most people fail at many job interviews before landing one they get, but is that because of variation in the performance of the interviewee or because of the differences in interviewers?
I would like to see a different analysis: for multiple interviewers of the same candidate, how similar are the ratings?
It also doesn't tell me about your personal value system and whether you are passionate about creating good user experiences or if you are just interested in a technical playground or if you are a social climber or what.
It also doesn't tell me if you are highly opinionated and want to do things your own way or if you are flexible enough to deal with the imperfections of the real world. I don't know whether you value the mantra that the perfect shouldn't be the enemy of the good.
No, none of these brain teasers or abstract bootcamp style things tell me that - and those are the actual things that will make or break a company and determine whether your product will hit the market on time. Those are the things that will get you a customer base and will create a company culture of productivity and not something that doesn't lead to results.
Until interviews become more about how people approach work and their ethic and process of conducting it (which is a major part of classical engineering), these are ineffective tools for finding the right people.
---
[1] although as a disclaimer, I strongly prefer companies of under 20 employees. If you have 200, then the employees can be more insulated from the exposure to these types of risk and the necessity of this type of vertical thinking. But it's an "insulation" - everyone is working either directly or tangentially on something that is intended to be used by something in the world (usually humans).
I'd be surprised if you're saying this as somebody who doesn't have a track record of working at big name companies who went to a big name school, but you probably are so.
I liked the interview approach of my last job - they had me debug some simple code. This was C++ for gamedev, so they showed me - a class violating the rule of 3 leading to a double delete, some multi-threaded code without memory barriers - or even volatile - and another example that escapes me at the moment. I was able to blitz through that. I floundered slightly when asked to explain the observed behavior of the buggy code pre-fixes - the exact behavior of the multi-threaded code before fixes surprised me, and I couldn't figure out the probable cause in my own head - but that segued easily into a discussion of the disassembly (at which point I was able to explain the behavior) when I suggested I'd look at the disassembly when asked how I'd go about finding out.
They asked about basic usage and design on a broader range of topics as well - I failed to remember to store a salt (doh!) for the simple user DB example schema, and floundered a little on my 3D math skills (in what to apply to solve a problem, not to recite the exact math - an area I admitted up front I was weak in) but otherwise did okay. Also a lot of discussion about previous problems I'd tackled and how (I've got a file of notes I keep called "War Stories".) I ended up getting the job at the price I named - they didn't even try to negotiate me down, perhaps I should've asked for more? :3
My point is you need smart enough people for your job and nothing more. You do an IQ test, a personality one and that's all. The rest can be learned.
It takes years to figure out patterns (when to use and not use them), good coding practices, avoiding newbie mistakes like sql injection and xss and how to actually make a deadline (its not easy to finish a project when you also have to learn the language and other skills in addition to the task at hand).
As a business owner, I don't want someone learning the above on the job and on my dime.
There is a company in MI that tried this called Compuware back in the 90s. It was a complete and utter disaster.
Anyway the interview process is so broken that they could actually improve it by just flipping a coin.
In fairness, I do think that someone who is very comfortable with a language or framework will be able to do things much faster than someone who isn't. Considering how quickly people change jobs nowadays, I can also see how companies don't want to train people only so they will leave shortly after or even before they're useful.
Obviously this is just based on my own experience in the US.
The only conclusion I reached is that one should navigate in life in such a way that your well being should not depend on people that are less smart, less experienced, less curious than yourself.
Someone who knows the ins and outs of their language won't have to look up or learn new stuff as they go for example. This isn't even necessarily restricted to languages or frameworks, if you've done something similar to an assigned task, you would likely have already encountered certain pitfalls and know the best way to complete the task.
We've reverted to the mean in recent years unfortunately. No training, and lots of nickel and dime bullshit. Today, if I found the right role, it would just be a matter of finding the right number.
Personally I don't think companies should be paying for specific training because as a developer you can learn anything you need online.
On the other hand, my experience is that many/most companies are willing to hire a senior developer even if they don't have experience with the company's specific stack. They rightly assume that any developer can pick up a new stack quickly enough.
I'm not saying this is perfect, just that's why experience is sought after. We have an annoying problem of way too many job ads being for seniors and very few for entry level. People have to get their foot in the door to ever become senior.
FWIW I agree most companies should train.
But I also understand how a lot of business owners feel about it. It takes time to learn these things vs someone who has basically already done this project before in this stack before. A business in a hurry (most of them) would rather have someone who already knows how to do it.
Look at Stack Overflow Careers. Look how many more ads there are asking for senior devs.
As an aside, it's paradoxical that the industry simultaneously desires 10x seniors, while prizes youth (who they don't have to pay at senior salaries). So the true profile of a unicorn coder is basically a 10x dev with 10+ years of experience who is also a new grad that requires no training. Keeps down the cost, all around.
Even jobs that don't really need developers of that caliber will hire like they do. I need someone to yet another billing app? Let me just hire one rockstar genius and squeeze the life out of them instead of two regular developers because it's cheaper.
If that is the case, what you need is a contractor, not an employee, and you should hire accordingly.
Google wouldn't hire someone to manage their datacenter with no previous experience managing anything like it.
That's exactly what they do. The interviews test for basic skills — an understanding of TCP/IP, unix, programming ability and intelligence — and all the rest is on-the-job training.
Company culture matters. Even the most intelligent people that are willing to grow will fizzle if they are in an environment that isn't productive.
I do not understand why interviews are not conducted in this manner. Pick some arbitrary idea and have the candidate work with the interviewer to design something, then let the candidate do a little implementation research/design, review with the next interviewer, wipeboard code and review some part of the project with next reviewer. Have a coffee break at some point to get to know the person. Treat the interview process like a normal day at work.
It truly is crazy.
It might work well for a small company especially with a small number of candidates but it doesn't really scale, unless you are a really big company (googleish size) that can dedicate so many resources to the interviewing process.
Doing design and research takes time, you can't do it in 30 min, you can't also ask someone to spend 8 hours interviewing, and if you have 10 candidates you can't spend half a man month or more on interviewing them either.
I've seen some companies that "interview" by contracting candidates for 1-3 days of work and paying them for it, but it really only works on positions where you hiring effectively interns/fresh graduates, but as most candidates especially those who end up in the face to face phase are currently employed you can't really do that.
After you've run this process enough times, your interviewers will have a good grasp on the various solutions and technical problems. Most people's training on how to interview comes from how they themselves were interviewed - which was probably poorly done. Improve the interview process and you can train people to be better interviewers just by being interviewed. :)
That might be one reason for the impostor syndrome. Because if we interviewed six people, one would be unqualified, four would be OK but not spectacular, and one would know the subjects better than anyone. So in an interview of six people the best qualified one of the six would have the job. The average four were not truly impostors, but would not get the job unless they had an inside recommendation or if we were really needed someone and there was a shortage of candidates coming in.
One wonders if it's possible to devise some sort of neutral, yet helpful, assessment that could be given to applicants without fear of legal retaliation. Or, if eventually an applicant will sue a company for not providing feedback after they were rejected.
It's my understanding that this is not the case.
People typically think they're above average.
https://en.wikipedia.org/wiki/Illusory_superiority http://www.livescience.com/26914-why-we-are-all-above-averag...
Glad to see that once again Dunning-Krüger proved wrong.
It's not even clear (to me) what's being compared here. What's the "actual performance" that the blog post very carefully does not define? The screenshot highlighting the technical performance question seems to suggest that it might be only that one point out of the three that interviewers fill in.
But it's completely possible to be technically competent and still feel that one didn't do well overall because one came across as nervous or boring or whatever. In other words, the comparison here seems to be between apples and undefined oranges.
- some mock-ups from when we first designed our product gallery
- a json file that listed the products names, pictures, description, etc.
- a blank css file
- a html file that loaded the css and jQuery, and then made an ajax request to load the json file. (This was back when jQuery was pretty much state of the art for front-end.)
With all of that in place, I'd explain that I'd like them to work on building a product gallery that looked similar to the mockups. I would further explain that, since we only had an hour, I didn't expect a polished or even fully functional product, but I'd just like them to dive in and see how far they got. They could use their editor of choice and ask any questions they wanted.
I felt like this gave better insight to candidates skill and workflow. My colleagues all gave more traditional interviews, so I also felt like there was a good counterbalance if someone didn't do well with this kind of interview.
Skewing interview performance based on some specific previous experience might be what you want if you expect your candidates to hit the ground running, but might not be the best to gauge long-term performance.
As far as the product gallery goes, I wasn't looking for a pixel-perfect product. Just spit out a bit of html and matching CSS that look vaguely like the mockup. The mockup images had things like the hex values of the colors written on the image to make that part easy.
Basically, it was trying to focus on things that would be the main part of the candidate's job there. The folks that we did hire all did well on both that interview and long-term at the company.
Actually we don't employ a woman (I'm working at a 3 person company). And we actually hire, but the only woman will probably decline on their side, she actually has experience in things we actually not need yet, so I guess we can't provide her a new challenge. way probably the best application.
Unfortunately we don't have the raw data available; but all we can infer from these heatmaps is that yes (and exactly as one would expect) there's a bit of variance between between the two sets of ratings (in both directions). Maybe a bit more than can be explained by the sample size, or from the simple fact people (on both sides of the process) are forced to discretize their assessments (so even if both parties were to objectively agree that the "real" performance was in the 2.4-2.7 range, which it is a good chunk of the time.... inevitably you'll find them be off by +/- 1 in their quantile assignments a good chunk of the time -- which is exactly what the data seem to show).
In particular: independent of how we explain the variance (whether due to sampling effects, or bona fide DK/IS) note that the vast majority of the variance is in the +/- 1 range. And on the alleged DK side (-2 variance) the measure is apparently quite miniscule.
However, that's not the original D-K effect. What D-K specifically predicted (and were able to show with much better data and methodology -- in their very specific sample population, at least) is that, among the general population there will be a striking degree of -2 variance, specifically in the bottom quartile. Going by the oft-quoted abstract from their original paper:
Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd.
In other words: "Performers in the bottom quartile estimated themselves to be performing, on average, at mean performance in the third quartile."
So if were to look for something analogous to a D-K effect, we'd expect to find a "hot spot" (or at least the strongest amount of red ink, for that quartile) in the upper left of the first chart.
Instead we find most performers in the bottom quartile (correctly) identifying themselves as in the bottom quartile; with a smaller portion bleeding up into the second quarter; and an an almost negligible portion placing themselves in the D-K zone -- namely, the third quarter.
If anything what the data show is that (even assuming there's nothing to contend with in how the actual data were taken), while inevitably there's some variance (both ways) in the two sets of rankings, and probably some of due to distorted self-perception -- on the whole there's actually much less of a D-K effect among engineers than among the general population (or rather, "Cornell undergraduates taking psychology classes" -- which, lest we forget, were the actual subject class used in the original D-K study).
So basically the opposite of what the author of the original article is claiming to infer from this data.