Hacker News Reader

Comment by grue2 | Hacker News Reader

I use a work sample (short paid contract), but it's not perfect and boy is it labor intensive.

As a hirer, you really can't win. There's not one hiring process nor guiding principle that doesn't seem to bring out the pitchforks of those who were frustrated, disrespected, or rejected by said process:

- Show us your open source work. (You're excluding all but a lucky few who have the privilege of writing open source code!)

- Okay then, show us a personal project or some work you've done in your free time. (What, so I'm expected to live eat and breathe code 24/7 to get hired?!)

- Well, how about a short contract/work sample? (How am I supposed to find the time to do that? I have a day job and a life!)

- Shall we try whiteboarding/coding tests then? (This is so insulting! Solving CS puzzles isn't what the job is about!)

One cannot, sadly, rely on the résumé. I have interviewed multiple self-deemed "experts" in such-and-such language, only to find that they could not even write a basic for-loop on the whiteboard.

View on HN

I mostly agree, but he still makes some good points. For example, "Give your interview to current employees".

One of my coworkers and I often joke about how we'd never pass our current hiring test, and yet we're among the highest paid and most productive people at the company. I've expressed this to the hiring manager and he still thinks it's a good approach. They have literally said they want more people like me... so they brought me into the hiring process, but I disagreed with them on almost every candidate, so they brought me back out of the hiring process. Some people don't know what they want out of interviewing a candidate, they just go with the status quo interview and assume they're going to feel good or bad about someone based on their results.

Most of these interview tests are game-able, there are a few resources you need to study hard for and you’ll be able to pass the interview questions without many problems. This has the perverse effect of raising the bar since everyone is doing so great on the problems and filtering is needed! But in the end, you only select for who is better at studying for the interview, not who is a better developer. Fun stuff.

If you onboarded in a time before this, or you aren’t practicing for an interview, it is very likely you wouldn’t just pass the interview, like you wouldn’t likely pass a physics college exam right now even though you did when you were in school.

I've found that it's usually obvious when someone is gaming my interview. Their answers are too correct, and too smoothly given. Those candidates are very quickly rejected and the recruiting agency basically told to scram.

Perhaps, but here's a personal anecdote when I was interviewed years ago. The interviewer asked me one of the trick problems popular at the time, which I knew the answer to. Ironically, it was not a problem I'd studied for; I just happened to have seen in elsewhere years prior, found it interesting, so I remembered it and its solution. To be clear, I didn't solve the problem the first time I'd seen it, either.

Nevertheless, I gave the interviewer a detailed, step-by-step, golden answer she was looking for, and she's obliviously impressed. After a few minutes, however, youthful stupidity^W^Wmy integrity got the better of me and I told her that I'd seen the problem before. So I got a 2nd trick problem, one that I didn't know nor studied for, and naturally didn't arrive at the golden solution.

I didn't get called back.

If you were to ask me to solve a problem that's on my study list today, believe me that I've got the selling part down pat now.

If I study a bunch of interviewing problems really hard, I’m bound to do good on your test. I can study hard, I proved that in unveristy and grad school. Your interview process probably isn’t that unique, and for bigger companies like Google there is plenty of study material out there.

And everyone is doing this now, it just isn’t a certain few bad recruiting agencies and applicants. Heck, even the good applicants (the ones that you want to hire anyways) have to do it to get through the loop.

It's almost as if the people who want the job the most would study for it the hardest.

Best company I've been hired at did a sample project.

They had the project all setup and ready to go. All I had to do was fill in some of the more complicated bits (hooking up to the database using LINQ to SQL, and implement some queries to search by name/album and two or three other fields).

Then display them in a certain order.

I was given an hour to do it. Apparently, after I was hired, I was told I was the only one they interviewed that could do it. Ironic part was I'd never used LINQ to SQL before and had studied it the night prior. (It was on the job posting)

Was it marked as a requirement by the job posting? Because if not, that's just a test of knowledge, not ability.

The test is the ability to apply knowledge, which is probably the only thing they need.

The commenter did say it was mentioned in the job listing.

Yes, I understand that it was mentioned, my question was whether it was mentioned as a requirement.

If the listing just said "LINQ-to-SQL knowledge a plus" but then gave you a test that actually required foreknowledge of it, then that's kinda shitty.

At my last job, I set up something like this. We were specifically interviewing people who claimed to have frontend and Angular experience - I confirmed this on every resume before the interview and again with the candidate before explaining the test. To test it, I gave the test to 2 coworkers who each completed it in under 15 minutes. Candidates were given an hour for it.

The test was a JSFiddle pre-filled out with Angular boilerplate. It had comments in places where things needed to be added and filled out. Just to be clear, there was no gotchas or tricks involved. It was very simple: "get JSON from this public HTTP endpoint, display it on the page in a table and then add a filter field. If there's time, add some CSS to make it look pretty."

Candidates were encouraged to use docs and Google if they couldn't remember off the top of their head how to do those. More than one candidate copy/pasted from Stack Overflow during the interview and while I wasn't impressed, I didn't count it against them.

Several candidates completely failed at it and I suspect it was a combination of nerves and lack of experience (one candidate it became clear that he had never used Angular before and he never got past the "get JSON from an HTTP endpoint"). During the test, I was more than willing to talk through the code with them and help them figure out the best way to do it. The one candidate who passed it was hired and they were very successful at the company.

Shall we try whiteboarding?

I've used this in the past. But, I don't ask for actual code. I'm using it to see the candidates thought process.

More important than any psuedo-code is how well they communicate what they are thinking (whether that be trough writing on the wall or conversation). Can they break the problem down into workable pieces? Do they ask questions about the requirements? Do they try a simple solution and iterate on it? They they get hung up on small details and miss the bigger picture?

You mean... "Hmm, my assumption, from looking at your resume and portfolio, is that you aren't quite ready to hold your own in a spontaneous, 1-1 discussion about an actual, real business or technical problem. And not only that -- I actually have severe doubts not only about your relevant experience, but about your basic intellectual capabilities. So let me give you this little made-up toy problem, with this little checklist in my mind about where I think you're going to screw up. And watch you perform. Being as I can tell you don't have a lot of options right now, and will put up with pretty much whatever I dish at you."

That might be OK for someone with, say, <2 years experience out of college. But for anyone mid-career or higher, it comes off as silly and condescending, and as not exactly a good use of their precious, irreplaceable time. And more to the point, conveys the exact opposite of the message you should be sending: that you know they're smart and intellectually self-sufficient, already. And that it's up to you to win them over, and convince them that you're worthy and interesting, and that they should jump over, and join your mission.

I generally agree, but I was involved in an internal hiring interview where a candidate was extremely qualified. In that situation, I was able to ask a hard and open-ended problem about graph traversal on the whiteboard. And it was fun to see the interviewee's approach. "Wow, I haven't done anything like this in years," was the reaction.

Obviously that is an example of a low-stress interview. When the candidate was stuck I gave hints, and the whiteboard portion was only intended to last 10-minutes. It was also only 5% of the feedback I gave. The majority was comments on candidate's knowledge and experience.

---

That's definitely how I'd like my whiteboard interviews to go. I've experienced the back-to-back grinding interviews of softball problems to write in syntactically correct code. Getting dinged for errors and even getting a close but wrong answer is a pain in the butt. When you write a program and realize it's wrong at the end, the question, "What do you think the right answer is?" is frustrating. I can't say, well, I wish I had 30 minutes to start over. If you tell me the answer, we can talk about why I was wrong.

Those interviewers have been engineers peeled off of their daily projects though, so I can't say I fault them individually.

WTF? Maybe I didn't communicate it well, but the whole point is the discussion, not the answer...

More important than any psuedo-code is how well they communicate what they are thinking

It's not a test of technical skills. And you'd be amazed at the number of resumes that signal somebody can have a quality discussion about technical problems, but the candidate can't communicate for shit.

But the candidate can't communicate for shit.

You know, it just might be... the interviewer who's not so hot at communicating.

Or is "communicating" okay enough, but following a template that's broken to begin with - and pretty much designed to make candidates feel alienated - or at best, highly unenthused at the prospect of playing along with.

Interviews are naturally awkward.

If you had that attitude during one of my interviews I'd show you the door.

I can't begin to count the number of impressive resumes that lead to unqualified candidates.

If you had that attitude during one of my interviews I'd show you the door.

And with that attitude on your side -- "if this isn't going well, then by definition, it's the candidate's fault" --- let's just say we're clearly not match.

I can't begin to count the number of impressive resumes that lead to unqualified candidates.

Ditto for job ads leading to unimpressive companies with obtuse interview processes.

Interviews are naturally awkward.

As currently conceived. So maybe it's time to start thinking about them differently.

You're awfully full of criticism today. In your perfect world, what would the interview process be? How do I, the hiring manager, ensure you are what you claim to be, without wasting either of our time or hurting your feelings?

Yikes. So which of the other 3 listed methods do you prefer? Or something else?

Also, I think you’re somewhat confusing whiteboarding and being interviewed by an egomaniac. If someone is a total jerk, I imagine any interviewing methodology will suck.

Sysadmin/infosec guy here. We do the same thing. I was shown this a while ago and I use it on other candidates I've interviewed. Propose a problem, then as they start working through it, throw other wrenches into it or state that their solution didn't work. See what tangents they go off to.

Another neat one I was shown was, as a whiteboard test, was to write down all the letters, A-Z, then come up with the name of a command for each letter. It's a double-sided test -- it shows your ability to respond quickly, how you respond, and the ability to back up your responses.

Of course, joking questions like "vi or emacs?" can be good as well to see a candidate's character as to how they respond to technical inside jokes like that.

Not sure about the alphabet. Got to K before I got stuck and thought... So in that moment, if I forget that there's kill that starts with a K, why does it matter? How does it relate to the fact that I know to use kill for sending signals? The recall for a situation where you'd use something and for a letter matching game seem to be very separate. (On the other hand I remembered join which I used maybe once in my life...)

It's multi-faceted. It shows your ability to think quick and how you solve multiple problems. If you stop in the middle of the list and try over and over to come up with an answer, how will you react to a real problem? Or do you just give up when it gets hard?

It's testing more than just rote memorization. You also have to explain any commands you write down, what they do.

Disclaimer: I've written code outside of work for my own satisfaction.

> - Okay then, show us a personal project or some work you've done in your free time. (What, so I'm expected to live eat and breathe code 24/7 to get hired?!)

I have software I can show off to potential employers but I didn't write it for them. I'll admit that I got really lucky to both enjoy coding and to have the privilege to so, but I didn't have to give up my life in order to get it done. I play video games many more hours per week than I code outside of work.

If I had to choose between otherwise equally qualified mechanics, I would tend to prefer the one with a hobby car at home.

Whereas I would choose the one with the cleanest shop.

I think I'd prefer the one who tells me the potential consequences of not doing a repair, without me having to specifically ask about it (or pry it out of them) and then verify the answer from another source.

Dealer service departments always fail this test. They will never tell you that it is not necessary to do a repair. Some will even say "I don't feel comfortable letting you drive it off the lot like this." Then you take it to another mechanic that says, "Yeah, this might affect your fuel economy by a tiny amount. I wouldn't worry about it in a car this old, unless you plan on moving to California."

Great story, but can you make it relevant somehow to the discussion about interviews?

Rather than hire the person who continues to do his day job at night, or who maintains an organized workstation, I'd hire the person that actually provides value to the company.

What if you accept applications for 30 days, then write the name of each applicant on a card, put the cards into a deck, shuffle the deck, then draw cards off the top of the deck and make offers until you run out of open positions?

It isn't merit based, but it's hard to argue that it favors one candidate over another! And maybe it's mimicking the effectiveness of your current system!

This has the added benefit that you only hire lucky people. Noone wants to hire an unlucky person.

This has a certain genius to it. The same genius as sleep sort.

This! We have tried all the points listed there and at least one person has found fault with that. That said, we follow pretty much the order you listed. The people invested in their job search do respond.

I agree that there is no "perfect" interview process, but I have seen from experience that there are ways to go about it that reduce the crappiness for both sides, beyond what a lot of "top" companies are currently doing.

How about sticking with standard phone-screen/on-site process, but emphasizing writing real code, on a computer, solving realistic software engineering problems? I've interviewed at several companies (including one of the "big four") that took this approach and found it was the best overall balance IMO. You get evaluated in a more realistic setting than a whiteboarding interview, but takes the same amount of time an also doesn't depend on "side-projects" or "take-home" assignments.

Some general examples from my own experience:

- Here's a poorly-written / buggy program: Find the bugs and do some refactoring

- Here's a loose spec for a small program: Spend an hour designing & coding it up (on your own, no one looking over your shoulder the whole time), and then walk through it with the interviewer AFTERWARD and explain your design / answer their questions about it.

I've learned that a decent proxy for this mythical silver bullet of hiring is basic questions about tooling and networking.

Gauging someone's vast, claimed experience in some language or technology can be tricky, but if they can't explain what a rebase is, or how maven dependency management works, or how SSL works in general terms, it tells me a lot about what kind of technologist they are.

As a preface, I’m not saying that those are inherently bad choices per se, because the underlying ideas are obscure enough that anyone who knows them at this moment probably knows other things you care about.

That said, never forget that ultimately your filtering criteria leak, and candidates will begin coming in with knowledge JUST about that, because you care.

Furthermore, you’re currently optimizing for trivia obtained in CS education that skilled programmers from other STEM fields and non-traditional backgrounds will lack. Besides people directly working on SSL, very few people need to know about symmetric encryption schemes at any practical level, and at best remember it at the same level I just stated it at, as a factoid. So you’re going to mostly find people who’d know that factoid and who’d remember that factoid; put another way, canned-answer spouters.

If you set up a web server for anything in the last few years you would have picked up something about SSL. Even if you just skim the rationale before copying the configs from Mozilla.

https://wiki.mozilla.org/Security/Server_Side_TLS#Modern_com...

If you used git, you probably even did a rebase. Or at least tried to. Or at least saw it as an option.

These aren't obscure university topics.

Right. They were chosen because experienced Java devs should be extremely comfortable with them. Not just familiar, but basically experts. And if they're not, it's a big red flag that casts a shadow over really anything else they have to say.

I'm sure each language/technology has a small list of universals that experienced practitioners should be quite comfortable with. Of course there are exceptions, it's not perfect, but my point is that these things are a good proxy for estimating overall competence level.

By what measure are you prioritizing Java+git over e.g. Java+Perforce in terms of universals?

Thank you for saying this! I have 30 years of paid CS experience, a degree, owned my own software company, worked for Fortune 50 companies and don't use any of those technologies. (I have used git, and used rebase, but like everything in git, it's so poorly named for what it does, that I don't remember exactly. I wouldn't even want to try to explain it at this point.)

And I bet those are exactly the kinds of things you're interested in and could answer. What about people who just aren't interested in maven dependency management (seems awfully uninteresting to me)? If your process is set up to find people who think like you and are interested in the exact same things you are, your process is broken.

And just to be constructive: you have a nugget of a good idea, that a technologist should have fairly detailed knowledge in some domain of interest. Finding what those areas are and getting them to speak about it beyond a superficial level of detail might be a good hiring criteria.

And that's one of the things done by companies in other businesses, who don't seem to have the same sorts of hiring problems that software companies do.

I'm an electrical engineer, and I've never been asked to design something from scratch in an interview. I have been shown existing design drawings or products and asked questions about them, which leads to various discussions. What we do in interviews is talk, both about what the company works on and about what I've worked on.

I agree with your idea of a proxy. Others seem to be interpreting it as asking trivia questions about your favorite pet projects and technologies. I don't think that's what you're doing.

In any given specialty, I think good developers will pick up the stuff around that specialty. Java devs will learn about Maven and git and maybe some JVM customizations to tweak memory usage. Rails devs will learn about which gems are widely-used to solve specific problems, and git, and maybe some devops/deployment stuff with SSH keys and Capistrano.

The particulars matter less than that there's some amount of holistic knowledge of their specialty. Of course one can argue the specifics (what if they've used svn but never touched git, etc)... but if there are no specifics -- if the candidate only knows their specific language of choice but nothing outside of it -- I would argue that they're probably not that great.

An alternate way to do this (which avoids false negatives cause they don't know the same trivia as you) is to ask them to tell you about a tool/library they are excited about. Or ask them to explain some part of their last project in detail. And then keep asking probing questions to see how deep they can go. You get the same sense of "how deeply does this person actually understand the tools they use" but you dramatically reduce the risk of hitting one of their (rare?) blank spots.

Side benefit #1 is you get a sense of what they think is important

Side benefit #2 is you find out if they can accurately gauge how well they know what they think they know

Side-benefit #3 is you might learn something new!

This is the kind of thing people in electrical engineering do.

As a candidate, I'm the most excited when there are short contract/work sample coding challenges that are problems directly related to the job I'm applying for, and I wish more companies did them than whiteboard stuff. It tells both parties the most information out of all of those options. To the employer: "Can this person do the job, or not?" Pretty simple. To the candidate, I think a coding challenge can tell you a lot about the company and what type of work you're going to be expected to do. I think it weeds people out much more effectively than whiteboarding. Making time for this part of thing is part of the job search, the same as scheduling an onsite.

On the third point, I think that of the four counter arguments, that one is the least convincing. If a candidate is applying for a new job and are asked to do a work sample or an exercise, but they can't find the time (generally exercises I've seen are between 3-5 hours long, and usually you have a week to finish them) then why are they applying for a new job? You could easily find 3-5 hours in a week to complete some exercise, even if it's broken up into multiple sessions of coding, if your goal is to work at a new company.

You are missing tests. Either short-timed coding or written tests.

The best way is probably a mix of some of those. Yet, I have no idea what mix :)

For what it's worth, we also think work sample assessments are the most accurate representation of whether a candidate will succeed (and there's a good deal of industrial psychology research that backs this up).

If you or anyone else is interested in a platform for technical work sample assessments, come check us out! www.headlightlabs.com

Reply on news.ycombinator.com