Hacker News Reader

Comment by swman | Hacker News Reader

I wish I could a 2-3 hour interview where I (or the candidate) showcase one of my projects and explain the architectural details and decisions in addition to showing any cool/hairy/insane code that got the job done. We can discuss those things and see how to improve them, or laugh at the crazy solutions.

Honestly how many times do I need to rehearse these dumbass algos (blah blah blah, so I'll optimize for space with blah blah blah) okay already. I would much rather show you real world code that I've built, or passion projects I spend my free time on. I want to bring me to your company/projects, so get to know what I'm about holistically as an engineer. I think you can best understand that by looking at actual work done and judging whether or not the person is capable of contributing to your needs.

Whenever we face challenges, we learn from them. At scale, we learn everyday. So just hire people who are passionate about facing challenges and learning from them. Not someone who can spend 8 hours a day like a college student playing leet code instead of building something useful. It really isn't that hard to memorize a dozen essential data structures and algorithms. But then what? So cringe.

View on HN

The challenge with this approach as that many very competent people are not allowed to discuss their prior work in that level of detail. More practical variants of this approach use a straw man software design problem to talk to that will exercise diverse areas of experience.

Yep, 100% this. Particularly if you've worked for big firms in the past, those NDAs do not mess around.

I do experience-based interviewing, and i have never encountered this problem. The great majority of the time, people are able to talk about anything. Sometimes, there are sensitive parts of prior work, but a candidate can just talk around those bits and focus on the rest. Even if someone had been working somewhere super-secret, if they aren't a junior, they have other experience to talk about.

If all your career experience so far has been at the NSA, yeah, you might want to do a side project before looking for work.

I’ve worked at some large companies and would say this is wrong. I can’t talk about most of my best examples because they’re still roadmap items.

You can’t remove the identifying details and talk about the technical challenge and usage in a generic sense? Your system or software is that specific? Can you give a now public example?

I can, and that’s usually what I do, but there are identifying details about the very specific domain which AWS only has a single product in.

So I can talk about architecting a system, about customer feedback and redesigns, but I can’t talk about work I did that will span another three years.

I think at FAANG it’s usually fine — most people have good enough examples even without their full repertoire. But at mid enterprise or F500 companies I could see this being a bit impactful.

> I can, and that’s usually what I do, but there are identifying details about the very specific domain which AWS only has a single product in.

Sorry, to be clear, I mean can you give an example of something you couldn't use as an example, but is now public.

I've worked at GAMMA, and agree, it's very doable to turn my work generic and talk about it in-depth, compare it to alternatives, etc. so that was the main driver in asking for an example.

it is not less wrong than timed leetcode BS interviews

As someone who's done classified work, the general guidance we got is that we can speak in generalities.

So I can't say what the code specifically does or who it's for, but I can talk about how I worked on a c++ engine wrapped in a Java server that was responsible for coordinating a large group of non-homogeneous assets, as well as various architectural details (what libraries did we use, database, messaging setup, etc). So it's not like I have nothing to talk about in an interview, plus the fact that I can't get too specific adds mystique. It's kinda fun, because in my experience there's an assumption most engineers/engineering managers make about what I've worked on from that first statement, particularly if they know where I work, and it's actually not that. :)

Also, I would find it weird if a person never worked on any side projects or tinkering in the free time to discuss the things above.

yeah, and some very competent people have no time or interest in timed coding tests and know that great software is not built that way.

if you have been through a CS program you have worked on probably at least 10 projects. and then there are your personal projects. if you have any experience you can talk about at least some of these apart from the stuff under NDA.

so, this approach is much better than timed coding interviews and is a much more fair system as it rewards and takes into account experience.

if you have been through a CS program you have worked on probably at least 10 projects

That's fine for a new graduate, but many of us are decades out of college. What we did then has almost zero relevance to jobs we might seek today.

the same goes for leetcode interviews. the key difference is that you don’t need to prepare to talk about your own projects. or at least much less. and you are probably working on stuff after hours anyway.

This follows what I think of as the best technical interview I ever had. It was for a more junior role, but it was pretty good.

Here's a basic straw man rails controller. There are a few things wrong with it. Apply suggestions. It was nice.

As an interviewee I'd like this too, but as an interviewer I wonder if it actually has enough signal. One of the problems I've found with these kinds of conversations is that people can plausibly BS quite a bit about projects, or their role in them.

Maybe I started a new compiler or something at my company but didn't have the chops for it and the project flamed out. If I lie and said that all my goals were achieved and all the hard technical challenges were overcome, how can you tell?

Or maybe the project did succeed but someone else came up with the idea and led the efforts. I was there for the technical discussions and grilled the lead on why he made the choices he did, so now I can answer your questions and sound like I know what I'm talking about.

Software engineers can make a lot of money, the incentives to game the interview process are high and people attempt it often...

Yeah, any member of a team that isn't completely clueless will be able to convincingly claim that all of the achievements done by the team was achieved by the person alone.

There is a hysteria that's plaguing the world of tech: The fear that incompetent people might "BS" their way to a position.

Everyone you talk with, has probably one or two anecdotal stories of such. "Yeah I worked with this CS grad that couldn't even write FizzBuzz" - yet we ignore the hundreds of other that do their work just fine.

And this is fought with setting up ridiculous 8-part technical interviews where you'll have to whiteboard some leetcode questions ("Given a problem, show us a working O(Log N), or preferably O(1) solution. You have 45 minutes") or design questions ("Show us how you would design slack / discord / zoom / etc.") that drags over 6 months.

For someone to be truly incompetent, or even not good enough to meet the company standards, the current system is overkill.

It's hard to tell if it's hysterical without knowing the Cost/Benefit for the company. How empowered are new senior hires, and how expensive is the time of the team? At larger companies senior engineers are often phenomenally expensive and at smaller companies controls over potentially business ending operations are usually minimal.

A friend of mine at a FAANG recently dealt with a bad hire. Their guess was that in the 6 months the bad hire was there they cost the company maybe ~5 million between wasted engineer time and delayed release schedules after people kept having to put out their fires. On the other end of companies, I've heard of seniors corrupting the database and its backup and ending an entire startup.

This is not a defense of whiteboard interviews per se, just an observation on why companies desperately want to avoid bad hires.

In your examples, did they go through a coding challenge / whiteboard process?

Across all examples I can think of - some yes, some no.

It's not material to my point either way. I was only arguing that for some company's being very afraid of bad hires may be sensible and dismissing their concerns as hysteria does not seem obviously correct to me.

>In your examples, did they go through a coding challenge / whiteboard process?

I think he was trying to point out that if some of the bad candidates had passed a whiteboard test, it brings in to the questions of the effectiveness of whiteboard tests to filter out bad candidates.

I remember a guy we hired years ago who answered all the tech questions we asked with flying colors. He was lazy and unmotivated and ended up dumping all his work on other people (me). I would have much rather had a motivated person who I had to teach some stuff to rather than him.

Saying this method of interview is not 100% effective so it's useless is not a great argument. Reality is that bad hires are really expensive (as argued above) and that current interview methods are somewhat - admittedly not 100% - effective at avoiding those while mostly - again not 100% - effective at admitting the great hires. Maybe this "convert to discussion" method is more effective (I have my doubts), and I'm sure there are more effective methods out there and we should look for them, but let's recognize the reasons for what we have while we look for something better.

>Saying this method of interview is not 100% effective so it's useless is not a great argument.

Who said that?

Honestly, one of the safest and most powerful tools is just to continually ask "why" questions as a result of whatever they say in regards to technical details, with a scattering of "how?" questions.

Others in the comments have expressed some legal concerns about "discussions" instead of Interviews, but I feel this is a hyperbolized fear based off a laypersons interpretation of US hiring law. The law requires you ask the same set of questions of all candidates, yes, but it reasonably understands that questions on past projects/work of course will never be the same, and these are legal and valid areas to go off-script.

So suppose in your situation you have a candidate and they talk about the project they worked on. To hear about their contribution, you ask (as you would all candidates):

"Tell me about a problem you encountered on this project that was particularly challenging from a technical perspective"

The candidate will tell you and likely their solution, and then you can start with the why's and how's.

"Why did you choose this approach? What are the benefits?"

"How did you arrive at this conclusion?" (you can further restate the question with "explain how you got to the idea that you needed to get here")

Make it a hard rule for yourself that you must be able to build the entire timeline and understand the candidate's role in that timeline. I find that even if it turns out they mostly were receiving orders from a more senior resource but could reasonably explain in their own words now why they ended up doing what they did, this is still a good candidate for me as they used available resources and took away a good lesson. If there is ambiguity, you can even just ask "why do you think your senior colleague had the right idea here?"

The idea of a discussion versus a checklist is you want to learn how a person thinks and how they approach issues. Even as a follow up to more factual check-list questions, "why" is very important as it helps you to understand the current state of the person. Just be honest with yourself that not everyone will have the same background and be honest on whether the "why" is relevant for the position. For example, some trivial linux kernel knowledge I would consider "nice to know", but it's by far not essential unless they're doing specific dev work on that field or claim to know it.

The other big catch you need to train yourself for is accepting that people will do things far differently than you do, even things that you consider wrong, but they work. The important part to focus on isn't about how close their answer is to yours, but to make sure you understand how and why someone reached that conclusion.

Why and how are very safe and legal questions as a follow up to a very straight-forward technical question.

> but I feel this is a hyperbolized fear based off a laypersons interpretation of US hiring law

Making definitive statements about liability is not trivial. I can confirm that at some very large companies legal has banned this style of interview citing discrimination concerns. I'm not a lawyer, so I can't comment on whether they're call is more or less correct but it certainly doesn't seem settled.

I believe you can see in my quote I even wrote "I feel" ;)

Also working for a "large" US company and discussing with peers from others, no such banning exists. There is very clear training and a heavy restriction on who can perform interviews, but this is not the same as outright banning it.

> The law requires you ask the same set of questions of all candidates

Does it? I have never heard of such a requirement in a law (though I’m not a lawyer). That might be an employer’s decision on policy they use to ensure compliance with the law, but I couldn’t quickly find any law requiring the questions to be uniform across candidates. (It’s also a difficult set of terms to quickly and confidently exclude the possibility that one exists.)

I guess that’s a longer (and hopefully more polite) way of saying “citation, please.”

Sure:

https://www.eeoc.gov/employers/small-business/3-im-recruitin...

Screen applications consistently. Apply the same standards to everyone applying for the same position.

Effectively, you should have the exact same criterion for all candidates for the same position and avoid "on the fly" questions/too heavily customizing it.

The legal discussion on all of this has basically boiled down to that you must ensure you're going through the same process for each candidate with the expected variances based on their employment/personal history (as it's relevant to the position)

I may overstate the situation slightly with that statement, but the idea is that if you ask about a networking stack for candidate A, you should ask the same probing questions for candidate B also. (Consider maybe you know that candidate B doesn't know this stack and will look "negative" on a review to the hiring managers, but you really like candidate B for other reasons)

Ultimately the idea is less about the specific questions and more that Candidate A and B are both reasonably considered in the same fashion for all of the requirements of the position, and that you aren't adding/omitting elements that may influence the applicability of the candidate in any fashion.

So let me restate: It's not required you ask verbatim the same words for each candidate (this is the lay-person over-read).

But for a given position, it should be established in advance a series of competencies that are necessary for the position and that are to be asked of each candidate. How you go about investigating it may vary from candidate to candidate, but should two interviews be reviewed, it should be expected that both interviews cover the same topics to a reasonable degree. (e.g., it is reasonable that if you ask "have you ever worked with library X" and the candidate flat out says they cannot answer questions on library X, you have reasonably covered that topic with this candidate. If you ask another candidate and they do have experience with library X and you ask some additional questions, you have not violated the spirit or letter of the requirement in this case.)

This is dramatically more complicated than it actually is. It does come back to competencies and structures. For instance, at AWS, this boils down to wide technical competencies and leadership principles. Of the nearly 100 interviews I did at AWS, I probably never had two that were very close, just due to people’s resumes, prior experience, and breadth of interests.

But, if you wanted to ask the recruiting team to add an interview to get more feedback, this would be a clear no.

I wonder how much of the last point is a high tolerance for false negatives? If a candidate is that close to a no-hire, maybe a rational choice is to say “No, you’re welcome to re-apply in N months.”

It was always cited as a legal risk.

Stripe handled this by having an extra interview baked in by default. People who were viewed as no risk hires didn’t have to do it, but the default was an extra interview to cover risks. Because it wasn’t an aberration, it wasn’t giving a candidate preferential treatment.

I honestly think it’s one of the smartest interview loops I’ve seen. I think they are one of the few large companies gambling a bit on the false positive side of things to shave off false negative points. And honestly, I was impressed with the people I interacted with at stripe more than AWS. They’re doing something right.

Reading this again, it was more confrontational than I wanted it to be and lacked details about why I think super structured interviews are a bad idea.

Some of the best candidates I’ve been involved with hiring have been compelling for reasons you’d never get to in a very structured way.

For instance, a candidate who was just out of university. They were very shy about technical topics, and their tech depth wasn’t great. They went to a university in Africa that I knew taught people more stuff that was wrong than right, and they were starting at a disadvantage. To get him out of his shell I asked about things he’d done outside of academics. He told me that he had not done much because he was very busy. Busy with what?

Oh, just starting a real estate investing side business with his family where he’d been responsible for their international expansion into the UK. With no finance background, he’d spent his time understanding the financial, legal and tax implications of their expansion and successfully launched in his third year of university.

And I’d done a bit of finance so we talked about concepts there and he was able to walk me through complex topics in simple terms.

My job for that interview was judging bias for action, curiosity, and diving deep into metrics. He couldn’t show that for tech because he went to a horrible university and had been very busy with other extra circulars.

He apologized to me for spending so much time talking about non tech subjects. And here is the problem: people who are bad at interviewing are punished by rigid processes. There’s a lot of data to show that women and immigrants from certain backgrounds are more reserved on average. You have to be a much better interviewer to pull off a structured interview that lets these candidates shine.

> Their guess was that in the 6 months the bad hire was there they cost the company maybe ~5 million between wasted engineer time and delayed release schedules after people kept having to put out their fires.

I've worked with one or two of those, but I've also been on interview loops that rejected candidates who went on to massively successful careers at a different FAANG.

The cost of false positives is relatively easy to quantify, the cost of false negatives significantly harder. Which is higher?

There is also a cost to hiring folks via an interview process that does not clearly signal, “we have thought about it and decided you belong here.” It reduces their willingness to ask for help and admit ignorance.

companies think that during interviews it is cheaper to let a really good candidate go than hiring a bad one. that is a huge mistake. every now and then they will not hire a key unique candidate, so they miss out on billions of $ in value. they don’t understand that if you hire 100 people, only about 5 of them will be doing all the important work and creating all the value.

> It's hard to tell if it's hysterical without knowing the Cost/Benefit for the company

I'm always curious how people interview doctors. As bad as getting a bad programmer might potentially be, getting a bad doctor must be exponentially worse, so you'd think they'd have vetting the unqualified folks down to a science by now.

In most countries doctors and other health professionals are registered and licensed, and must submit to strict examinations.

Mistakes can also be career-ending and potentially result in criminal prosecution.

It's way harder to get hired as a doctor than "do a bit of leetcode on a whiteboard".

Exactly. Somehow there is this impression that software engineers are having it really hard and they face worst job requirements/interviews.

I'd imagine software/IT might be only place where people who have delivered a couple of toy sized, half-assed webapps are now experts commenting on 'engineering challenges' and 'industry trends' with profundity.

> "I'm always curious how people interview doctors."

They go through several years of medical residency under supervision before becoming a full fledged doctor and are burdened by expensive liability insurance and ongoing education requirements. That takes care of the competency beforehand.

I think about the time my manager dismissed an interviewee as not understanding map-reduce because he couldn’t solve some problem in 2 passes, but rather needed 3.

I asked him to explain how to do it in 2. And he was like, “what? It’s easy!” Then another person on the team who over heard it also asked.

Eventually the whole team was there, asking him to prove it, and he spent over an half an hour trying to explain it, and none of us were getting it. The consensus from everyone (except the manager) that it was an unfair question that relied on a a very subtle assumption about the data that wasn’t obvious at all, and that 3 passes was the minimum required Without the that assumption.

Of course the manager said everyone was stupid and it was a good question.

That manager remains my counter example of what a good manager is.

this example display the advantage of leetcode. if your recruiters just can not work, leetcode style still can offer some useful data for decision.

An awful lot of the datastructure-style questions rely on specific knowledge of the optimal approach. I dislike asking questsions because they are more a test of whether you took a particular style of datastructure course than a test of problem solving

Sadly this isn't some sort of mass hysteria but based on practical experience. Yes, it's hard to believe. New interviewers are routinely shocked the first few times they are asked to take a candidate through a coding test. That's why everyone should just ignore the advice in the article - it's wrong. If you want to hire competent programmers, you need to test them rigorously by watching them code, in front of you. Every time I have been tempted to stray from this path the results have been bad. The world is full of people who are very good at seeming affable, friendly and competent but who then fall to pieces the moment you ask them to write a program. Any program. That does anything at all.

Then why ask candidates to solve a leetcode hard or two mediums in 45 mins? If you're afraid of people who can't code any program then ask for leetcode easy.

Sure but the article isn't about difficulty of coding challenges, or where you get them from, but about doing them at all.

It's a vicious cycle.

As companies ask 'Leetcode easy' questions, there came to be thousands of online blogs, youtube channels etc.. which trained even the n00bs who can't write good code otherwise. If they train very well they can solve 2sum or write binary tree level order traversal without understanding much. Of course not all interviews can use novel exclusive questions, and these have a non-negligible chance of passing.

Now if you are hiring, you would think, "If these n00bs can solve Leetcode easy with practice, the good ones are solving Leetcode medium with same level of practice. So let's raise the level of questions so that we don't end up hiring these rote-learning noobs". And it continues.

I don't think there's an obvious solution to this, if you don't want to lose the statistically good heuristic of problem solving skills, in order to weed out candidates.

I haven't actually seen that cycle in practice when I've been hiring. Maybe a small number of people practice a lot on leetcode, but it seems most don't. And I think it's hard to get better at those sorts of problems and not get even a bit better at programming in general. Remember the problem here is people who can't code their way out of a paper bag, not people who just struggle with exotic algorithms questions.

The glaring problem (IMO) with throwing hard LC-style questions at interviewees, is that it's a fundamentally stressful and high-stakes situation - which is so far removed from the actual working environment.

What are you actually inferring from the situation - how well the candidate can write software? How well they tackle stress and anxiety? How much spare-time they have to grind these types of questions? etc.

Probably a mix of them all, but there's a lot of irrelevant noise - if you're just looking for one thing.

I am an interviewer for C++ job candidates in the automotive industry.

I do a coding task first and a Q&A afterwards — because these are the requirements.

I did many interviews. My experience is: I could skip the Q&A completely. Most candidates could answer the questions just fine after reading the Wikipedia article for 10 minutes. In the interview I can see if they already read it or not. But I don't think it matters.

What matters are the coding skills. The coding task is quite simple and half of the candidates (with master degrees and 'years of industry experience') fail. But these candidates are often good talkers when they are chatting about the benefits of agile methodology and so on.

I totally agree with you that a candidate should just write any program. I believe I know after 15 minutes if they are developers or not.

But I also agree with the idea that we interviews must calm down the interviewee and remove the nervousness. I don't want to see if they stay cool in an exam situation because daily work is not an exam situation. I want to see if they can code when they are relaxed.

>half of the candidates (with master degrees and 'years of industry experience') fail.

If half the candidates with a master's degree in CS are failing your test, that should be a pretty huge red flag to you that your test might have an issue. What question are they failing?

They don't have problems with the questions. They are not able to write a simple C++. And they are applying for a C++ dev job.

I would think that could get filtered out on the resume check with no relevant experience. Are they lying on the resume?

I honestly don't know. I asked myself the same question.

These guys are applying for C++ dev jobs but are completely unable to write a program. Most of these fake programmers have studied electronics (not CS) or have studied in a country where you can buy a degree.

It's extremely common for people to not only advertise that they have specific skills, but even to have been using those skills for years, yet be entirely unable to demonstrate those skills when requested.

C++ is a particularly badly affected language for some reason. I think there are a lot of people who learned C++ once, a long time ago, then moved to Java as quickly as they could when it came out. But nobody likes to admit that maybe they lost a skill they once had, so they keep putting it down on the CV regardless. And then when asked to write something like hello world, they don't remember how to use std::cout or whatever.

The vast majority of people you actually work with are competent, sure. But the problem with interviewing is that it adversely selects for incompetent people, because competent people are more likely to have jobs and not be currently interviewing.

Other fields seem to do just fine with this approach.

If you interview for an R&D position at (e.g.,) a biotech company, in academia, or one of the National Labs, you're usually asked to prepare a 30-45 minute presentation about your past work. New grads usually use their MS/PhD thesis defense slides; other folks often have a conference talk they can expand. Some places also do "chalk talks" where you describe how you'd approach a new problem of your or their choosing.

This presentation is the jumping-off point for the rest of your interviews. If you talked about building a compiler, someone is going to ask for details about the lexer, maybe have you do implement a very simple tokenizer on a white board. Another person will ask how you measured its correctness/performance. Yet another may dig into how you organized the team doing the work, etc.

Maybe, as you propose, someone else helped with parts of the work. This wouldn't necessarily weed that out. Nevertheless, I'd argue that being able to justify the decisions that were made--and cogently explain when/why they might be different--isn't actually BS; it's understanding.

What happened to references? That’s an easy way to verify if someone is bullshitting. It is possible to bullshit your way through interviews (there’s a whole industry to help you do just that) and not know good software engineering patterns.

Most companies won't do references beyond verifying employment. Then you relying on professional references, and are they lying?

This is pretty pretty much what I tried to achieve here https://github.com/philbert/take-home-tech-test

The point is to have a conversation about a project that the candidate understands well and is passionate about rather than asking them a bunch of questions that we already know the answers to.

Before the interview we review the code base and try to understand what it’s doing by the documentation provided in the readme. During the interview we get the candidate to demo the project and any questions that came up in our code review we ask at the stage of execution in the project demo. The interview lasts for 2 hours and we’ve had several rounds of candidates put through this process.

Both we as interviewers and the feedback from candidates has been very positive. The interview ends up being a day-to-day normal experience within the team and this really helps us to gauge the team fit. I think this helps to hire people that compliment and expand our skill set rather than hire people who are basically ourselves.

I feel like this is a pretty big ask to do for an interview, unless you happen to have it lying around already.

Yep, agree. 95% of the code I write is owned by my employer and is under NDA various other privacy / IP laws. The 5% that isn't, has no place in an interview. It's a bunch of brittle glue code automating and backing up data between my devices.

"Passion projects" in my "spare time"... maybe once the kids have grown up and flown the nest...

Leetcode is easy... I memorise a bunch of stuff, do the dance and pass the interview. If i'm lucky I get a problem i've not seen before and actually have to use my brain during the interview.

Isn’t the time spent memorizing leetcode similar to the time spent building a side-project?

I took a look at leetcode when I was interviewing and decided it was a waste of time for me to learn that dance. I was happy with my chances with the companies that didn’t use it in their interviews. And it worked out fine.

No, it’s not, because take home assignments are not typically reusable. Learn leetcode and it is valuable at most companies you will apply to. It’s more respectful of your time.

I was comparing leetcode to sideprojects, not take home assignments. The parent company was also talking about side projects.

Side projects will help you learn useful skills and it is the suggestion of OP that it should be used by more companies in place of leetcode interviews.

Btw, a lot of companies don’t use leetcode for hiring. In my last job hunting season, I would guess than less than 20% of processes used leetcode. Pretty far from “most” companies.

Completely agree re the value of having side projects. I wish I had the luxury of time right now, maybe in a few years once the kids are older.

The leetcode dance is, at least for me at this point, much lower effort than starting a side project on the understanding the code I produce will be reviewed during interviews. It's like Sudoku, once you've done a "few", you get to the point where you're able to solve them quickly.

> Isn’t the time spent memorizing leetcode similar to the time spent building a side-project?

The nice thing about leetcode it is easy to bound the time to what you can handle. I do one puzzle a week and set a timer for 20 minutes. Then I get the answer and browse the forum. It's basically the equivalent of solving the Sunday crossword for me, and it keeps my algorithm skills sharp. Probably no worse than burning a lunch hour on HN.

I do my side projects the same way

> 95% of the code I write is owned by my employer and is under NDA various other privacy / IP laws.

True, but would your employer care (if they found out) if you copy/pasted a small portion of code that’s not considered critical IP (like util functions, and an integration syncing records from your backend the Salesforce, or something tangential to the business outside the core product)

May technically be breaking your employment agreement, but I can’t imagine it would be too hard to pluck out a decent amount of code, and re-write portions of it if necessary to “anonymize” it for interviewing purposes.

Or if you happen to be interviewed by someone like me, my approach is simply “if you can’t show me the code, show me the UI and explain how the backend / frontend works, and a discussion ensues.

> True, but would your employer care (if they found out) if you copy/pasted a small portion of code that’s not considered critical IP

What? Yes. I expect I'd be in prison for a few years. Your interview approach selects for people who don't follow their NDAs

Sergey Aleynikov went to prison over copying GPL code.

As mentioned in my original comment, if someone isn’t comfortable showing me the source code I simply ask for them to describe how it works, challenges in building it, how they built it, etc.

I could do this and would welcome this type of interview experience. I'd be able to talk in general terms about the problems I work on and their solutions but not the low level specifics.

To add some context, my work is in Financial Services, trading systems and whatnot.

> would your employer care (if they found out) if you copy/pasted a small portion of code that’s not considered critical IP

Who decides what is "critical IP" and what is not?

This is a terrible advice, and no, you should never copy code from a work project without being pre-authorized to do it.

It's not just common sense, but you also might trip up over unexpected legal issues (licences etc) that you may not even be aware of.

I would immediately end my interest in a company if I was asked to do this. This is far too general a problem, which will harm the interviewer as much as it will harm the interviewee.

Contrast it to giving a candidate some slightly broken code in a framework related to the role and then asking them to a)fix it and b)implement a new feature of their choosing and document it.

The advantages of the latter approach:

* The interviewer doesn't need to prep beforehand as they already know both the problem and the codebase. This means they can ask much more interesting questions and don't have to invest significant time in reviewing a project that might be in a field in which they have no experience. This in turn leads to better discussions which in turn leads to better interviews.

* The candidate is given a much tighter problem definition and isn't required to come up with something a)novel, b)not covered by their current employer's NDAs.

* The candidate's time is respected because they can be told how long it should take them up front.

* Each candidate gets a standardised problem and so it's easier to compare between them.

When I give take-home assignments, I'm just looking to quickly confirm that someone has a working home dev environment (i.e. they don't just code inside environments that other people give to them) and that they can understand a small codebase and write clean code and document it. Everything beyond that I can find out by talking to them during the interview.

When I actually used this technique in interviews it was interesting to see how many supposed senior engineers would reply with things like "I'm getting a %JAVA_HOME NOT FOUND error, please can you fix the repo and let me know when it's ready for me to work on?".

We've been using it for DevOps roles which are not highly specialised in any particular technologies and require ability to solve problems at a general level.

The test is intentionally designed to filter out candidates who cannot meet or do not want to meet the technical requirements.

Fair enough, maybe that works better in devops, but it seems like it will filter out candidates who have a good work/life balance.

> You do not have to start a new project from scratch. It’s perfectly fine to submit something you have previously created yourself. Maybe it’s something you work on in your spare time, just for yourself!

Yeah, no, nobody is or should be giving you their own personal work. It's offensive and probably illegal that you'd even ask.

> The novelty and creativity of your submission

Yeah, no.

This is a really bad take-home exercise, because it is way too vague. This vagueness makes it very hard for me to decide how much time to devote to it, and what task is complex enough so that you will consider it, but not too small so that you don't throw it away. Instead of giving me a bar to jump over, you make the bar invisible, then expect me to jump just above it. So I hope you do have a default project for those who are not "creative"... at least not for the purposes of a job interview...

Red flag test. Vague, no guidance around how much time to spend, no guidance around where to focus from a technical perspective (i.e. build anything/everything), bias towards someone who already has some related code completed. Given the lack of clarity, accurately assessing one candidate vs another would be difficult. It also comes off as lazy.

This is almost exactly how I hold interviews, except they are usually < 1hr. I ask a person to tell me about a recent project and then keep asking more detailed questions about things until one of us hits our limit of understanding. No gotcha questions or random trivia. I only ask questions about something they claim to have done. Occasionally I'll ask a category question like, "Have you done any work with multi-threading?" And if they say yes, I ask about that project.

I've found this technique to be extremely successful. It's possible I may have had some false negatives, but I've never had a false positive. Everyone I've recommended for hiring has been successful.

> I've found this technique to be extremely successful. It's possible I may have had some false negatives, but I've never had a false positive.

This is the impossible problem with hiring. Every interviewer wants to minimize false positives (they're expensive!), but every interviewee thinks they're a false negative.

Which is funny since I bet most who thinks they are a "false negative" also would claim to have "impostor syndrome". Anyone with impostor syndrome would view themselves as a true negative.

> I bet most who thinks they are a "false negative" also would claim to have "impostor syndrome"

I don't think this could be true? If you think you don't belong/deserve the job (imposter)... why would you think you deserved the job (false negative).

Everyone I've recommended for hiring has been successful.

It's possible you're good at spotting good people, and rejecting bad people, but it's also possible that hiring is just easier than you think it is, and most people are capable of doing the jobs you hire for. You have no way to tell if you'd have the same result just hiring people by picking random resumes. Maybe negatives are just rare.

It's unlikely that hiring is easier than I think it is, because I think it's pretty easy. The people who think it's hard are the HN consensus and the large corporations we keep reading about coming up with these convoluted interview standards/metrics.

Same here. I've never had a dud in terms of ability when doing a technical chat interview. People who don't know how things work will hit a wall in this format, it's not actually that easy to BS what your thoughts on the CPP memory model are.

The fear of BSers seems to be what holds people back from this approach though, and I can see that it wouldn't work for certain non technical fields.

> I've found this technique to be extremely successful.

That's just survivorship bias waiting to explode. Also, they're successful by some arbitrary metric that is applicable only to your company.

It's entirely likely I will one day hit a false positive.

I've been on the hiring team for every employer I've had for the last 15 years and have hired dozens of people.

I don't think that style of interview provides as much signal as you think it does. I would probably say that watching someone work through a problem provides more signal than having someone explain a problem they have already solved. That's why I feel that all of the companies I've been at, eventually ended up shifting the interview process to do less "explain a project" and converging more on "talk us though these different types of problems". And then the past experience is what is being demonstrated when they show off how deeply and quickly they can think through the different types and bring their experience to bear.

I also just generally do not think that most people can even put together a 2-3 hour presentation of their past work that goes over well. Most of the time people can't really talk more than 30 minutes about past projects. That requires a whole different set of skills, which there are certain contexts where I'd value that more highly. But my initial impression is that if we set up our interviews this way, we'd also end up filtering out a lot of people who'd be great, since not picking the perfect project to demo basically dooms the entire interview to be a flop. With multiple interview types, you increase the chance that there's an interview they really shine in.

I often administer a 45 minute version of this (talking about any/all past work, not necessarily side projects), as part of a half-day of interviews. We'd never be able to get into the depth you could in two hours, but we can get somewhere.

My experience has been that a lot of candidates can't even fill 45 minutes. I ask "what was the most interesting part?", "what was most technically complex?", "what would you do differently if you did it again?", and they just don't have nontrivial answers.

>and they just don't have nontrivial answers.

Yeah, because you can have a career where you're just gluing stuff together to make business apps for, usually, simple business problems. You're looking for craftsmen but you're interviewing plumbers.

To extend the plumber analogy, prospective employers will frequently look for plumbers with specific experience in copper pipe, rigid PVC pipe, or flexible PEX pipe, as though fragmenting the plumbing space in this fashion has any bearing on whether or not the result will conform to building codes, ensure that all the drains and faucets work as expected, and generally solve any fluids transport problems that may come up without having to push the calendar to the right.

Most people look up the local business listings, pick anyone advertised as "plumber", and call to make a service appointment. Or they use a general contractor that already has a list of approved subs. Master plumbers don't have to answer little trick questions about brazing copper or about finding lead pipes in an old building. People somehow trust them to know what their job is, and do it.

Rarely, one might encounter an unreliable plumber. They might not get paid, and any other plumber is usually able to fix their botched jobs without hurting the budget much. Review sites exist to track building-trades business reputations.

But the analogy breaks, because no one trusts software and IT folks to do their jobs competently. The default assumption is that we are all know-nothing hacks who could destroy the company with one keystroke. All our knowledge is assumed to be tightly siloed, and does not transfer between similar technologies. C++ people can't do Rust or Go. Java people can't do C#. Desktop people can't do the cloud. Back-end people can't do UI. CMMI people can't be Agile.

It's madness.

Quite often, though, you hire tradesmen and they're bloody terrible. More often than not. So i'm not sure this is a very motivating analogy.

100% give me a convo with a dev like this anyday of the week over this crammer whiteboarding l33tcode problems.

You can make it even more generic than that. A rather simple at first glance, but discussion provoking question: "What do you like and what do you dislike about the Technology X?". A good candidate who has experience with the said Technology X would not shut up on this subject. A bad one however will struggle. This is of course provided that you are hiring for some specific stack.

> experience with the said Technology X would not shut up on this subject

I've learned the hard way to be very reserved when giving opinions about technology X because interviewers sometimes get really defensive about the thing they like about technology X.

The bias in this approach is obvious. In your mind, you think a candidate "not shutting up" is a green flag, but the reality is you are trying to hire someone who thinks like you.

There are a million reasons why this approach doesn't work. Maybe they just don't like talking that much. Maybe they don't like the technology you're discussing. Maybe they just don't care enough to have an opinion on it.

None of the above are good reasons to pass on a candidate.

> but the reality is you are trying to hire someone who thinks like you.

Not really. If someone can present solid reasoning and argue their point well, they don't have to think like me. Three's more than one way to skin a cat.

> Maybe they don't like the technology you're discussing. Maybe they just don't care enough to have an opinion on it.

You mean the technology they are being hired for? Yeah, hating, not caring about it is probably not a good motivator to go for the job then, wouldn't you agree?

> None of the above are good reasons to pass on a candidate.

Beats the whiteboarding. I was hired like that in my last 3 jobs and I have used the same approach myself with rather consistently good results.

Oh, well it sounds like you're doing everything right. This also probably scales really well.

Thanks for sharing - hopefully a lot of people will use your anecdotal evidence to impact the way they hire people too!

I don't like it, but I get it. You can train for the algos, so just do what needs to be done and train for it. Maybe you don't like to do what needs to be done? Well, that filter worked.

Also, putting the emphasis on the interviewer understanding the candidates code instead of the other way around is never going to be popular ;-)

> Also, putting the emphasis on the interviewer understanding the candidates code instead of the other way around is never going to be popular

It's also hard to scale it, make it objective, and keep the efficiency high (most developers prefer not spending their time on either side of the interview table)

There is another advantage to the "algos". If you can learn algorithms then perhaps you can learn other things as well.

Every new job I've had involved quite a lot of learning in a very short period of time.

Learning whatever language, and whatever standard library is almost always trivial. They're usually not all that different, well documented, some with textbooks even.

Learning to navigate and reason about the huge number of undocumented, often arbitrary, system architecture decisions, design decisions, code layout, etc. etc. that make up real code bases.

That's hard. Really hard.

This is how I interview.

Show me something you've done that you're proud of. Take me through it in depth. Answer some questions on it.

Doesn't have to be a free-time project. If you don't have a free-time project and you're too NDA'd to discuss previous work, then present some language feature or something.

You can sometimes. This is literally one of the interview questions I ask - tell me about a complex project.

Reply on news.ycombinator.com