Hacker News Reader

Comment by fhd2 | Hacker News Reader

The last time I've used a leet code style interview was in 2012, and it resulted in a bad hire (who just happened to have trained on the questions we used). I've hired something like 150 developers so far, and what I ended up with after a few years of trial and error:

1. Use recruiters and network: Wading through the sheer volume of applications was even nasty before COVID, I don't even want to imagine what it's like now. A good recruiter or a recommendation can save a lot of time.

2. Do either no take home test, or one that takes at most two hours. I do discuss the solution candidates came up with, so as long as they can demonstrate they know what they did there, I don't care too much how they did it. If I do this part, it's just to establish some base line competency.

3. Put the candidate at ease - nervous people don't interview well, another problem with non-trivial tasks in technical interviews. I rarely do any live coding, if I do, it's pairing and for management roles, to e.g. probe how they manage disagreement and such. But for developers, they mostly shine when not under pressure, I try to see that side of them.

4. Talk through past and current challenges, technical and otherwise. This is by far the most powerful part of the interview IMHO. Had a bad manager? Cool, what did you do about it? I'm not looking for them having resolved whatever issue we talk about, I'm trying to understand who they are and how they'd fit into the team.

I've been using this process for almost a decade now, and currently don't think I need to change anything about it with respect to LLMs.

I kinda wish it was more merit based, but I haven't found a way to do that well yet. Maybe it's me, or maybe it's just not feasible. The work I tend to be involved in seems way too multi faceted to have a single standard test that will seriously predict how well a candidate will do on the job. My workaround is to rely on intuition for the most part.

View on HN

When I was interviewing candidates at IBM, I came up with a process I was really happy with. It started with a coding challenge involving several public APIs, in fact the same coding challenge that was given to me when I interviewed there.

What I added:

1. Instead of asking "do you have any questions for me?" at the very end, we started with that general discussion.

2. A few days ahead, I emailed the candidate the problem and said they are welcome to do it as a take home problem or we could work on it together. I let them know that if they did it ahead of time, we would do a code review and I would ask them about their design and coding choices. Or if they wanted to work on it together, they should consider it a pair programming session where I would be their colleague and advisor. Not some adversarial thing!

3. This the innovation I am proud of: a segment at the beginning of the interview called "teach me something". In my email I asked the candidate to think of something they would like to teach me about. I encouraged them to pick a topic unrelated to our work, or it could be programming related if they preferred that. Candidates taught me things like:

• How to choose colors of paint to mix that will get the shade you want.

• How someone who is bilingual thinks about different topics in their different languages.

• How the paper bill handler in an ATM works.

• How to cook pork belly in an air fryer without the skin flying off (the trick is punching holes in the skin with toothpicks).

I listed these in more recent emails as examples of fun topics to teach me about. And I mentioned that if I were asked this question, I might talk about how to tune a harmonica, and why a harmonica player would want to do that

This was fun for me and the candidate. It helped put them at ease by letting them shine as the expert in some area they had a special interest in.

Love the 3rd point! I might start using that in my future interviews

Please report back when you do!

I don't remember how I came up with the idea. Maybe I just like learning things.

One candidate even wrote after their interview, "that was fun!"

Have you ever had a candidate say that? This was the moment when I realized I might be on to something. :-)

Interviews are too often an adversarial thing: "Are you good enough for us?"

But the real question is would we enjoy working together and build great things!

People talk about "signal" in an interview. Someone who has an interest they are passionate and curious about and likes to share it with others? That's a pretty strong signal to me.

Even if it has nothing to do with "coding".

If this became popular, wouldn't people start rehearsing for it like they do Leetcode interviews, and it would be become another performance that people focus on and optimize for, rather than on the skills for the job?

Rehearsing how to teach something is not a bad thing.

And I'm sure you're going to get different questions about different aspects of the things you're trying to teach from different interviewers. The wonderful thing about this is that it models trying to teach a concept to a fellow coworker. How you handle the questions during the teaching time says a lot about the candidate.

But would this train people to teach well?

Or would it train people to choose topics very carefully, such that a little teaching skill goes a long way?

It's not like the interviewer getting a lesson in tuning a harmonica is going to bust out a harmonica and start putting his newfound knowledge to work, or revisit the subject in 6 months to see if he's retained the knowledge, or bring in a panel of harmonica tuning experts to check there weren't any major gaps or mistakes in the lesson.

I think the lesson to be learned from the parent comment is to put candidates at ease and let them express their interests. I think it doesn't matter if you chose to use "teach me something" specifically. However, it does matter how to try to be accommodating towards the candidate either by asking about their hobbies, some recent news any fond memory/project etc.

I forgot to include something important in my initial comment. In my introductory email, I explained that the "teach me" segment is completely optional.

If someone didn't want to do it, that's fine and wouldn't be held against them. In practice, I think one person out of 20 chose not to do it.

And if they weren't a great teacher, that was fine too!

The purpose of the segment is to give the candidate a chance, if they want, to shine at something they are interested in, and help put them at ease by letting them start out being the expert.

Of course. Campbell's law[1] applies.

[1] https://en.wikipedia.org/wiki/Campbell%27s_law

I think the point is to remove the adversarial aspect and therefore see how they react when not under pressure. I wouldn't be surprised if OP does warn their candidates about it.

You have it exactly right. I always told the candidate in my introductory email that the segment is optional and I like to learn all varieties of new things and they should feel free to pick any topic they want.

And then when they were teaching me, I made sure to pay attention, ask questions about anything I didn't understand - not to judge their teaching skills but to show I was interested and listening.

That sounds really cool. I wish I was running into more job interviews like the one you describe. The adversarial interviewing really hurts the entire feel of the process

Man, I really hate it!

I can't fathom how making the whole process being all hostile makes any real sense!

We all want that great person to show up and really help, not just cope or put up a facade to survive.

Yet, that is exactly what happens. People that other people expect to bond with get put through the wringer....

Seems counterproductive to me.

> The adversarial interviewing really hurts the entire feel of the process

Agreed. I have been through technical interviews where at the end, my feeling as a candidate was "I don't want to work with this asshole".

> In my email I encouraged the candidate to teach me something unrelated to our work, or programming related if they preferred that.

What if I decide to teach you something about the Quran and you don't hire me?

Perhaps this is just urban legend but from the stories I've heard hiring for FAANG-type companies there are people out there interviewing with their only goal being baiting you into a topic they can sue you over.

Worst instance I have heard of is when an interviewer asked about the books the candidate liked to read (since the candidate said they're an avid reader in their resume) and he just said that he liked to read the Bible. After not getting hired for failing the interviews he accused the company of religious discrimination.

I'm by no means an expert in US law and don't know the people this happened to directly so maybe it's just one big fantasy story but it doesn't seem that far fetched that

- If you are a rich corporation then people will look to bait you into a lawsuit

- If you give software engineers (or any non-lawyers) free rein on how to conduct interviews then some of them can be baited into a situation that the company could be sued over way more easily than a rigid leetcode interview

I think nothing came of the fellow who liked reading the Bible but I would imagine the legal department has a say in what kind of interview process a rich company can and can not use.

I have found that if you tighten the parameters a bit, you can still get all the benefit of what this question is asking for. For example, teach me the rules of a sport, board game etc. You still get to see if they can present a coherent explanation of something relatively complex, but you can avoid these potentially dangerous topics.

> What if I decide to teach you something about the Quran and you don't hire me?

Simple, don't pick subjects, such as religion, sexuality, etc. It's not that deep.

and like you said it's not that deep. pick something light.

a candidate that pulls the religious thing now is one that's going to drag their crazy shit into the office later. or complete misread the room / culture now, and in the future.

like it's a technical job interview, talk about something vague technical, even if it's just how the convection settings work in your oven.

Or, you can choose to not live in fear.

This is the way.

Secondly, share that place by example. Radiating non fear can do wonders for the people around us, as can honest and frank conversation.

When all members of a team, or even small companies come to understand not all environments are about blame and shame, magic happens!

People who are poor managers or in positions of power, but not emotionally regulated or unwilling to get therapy, find blaming others easier than self awareness and doing the work of bettering themselves. Couple that with financial success and why should they when all our society values is how big your bank account balance is.

Truth!

Don't have anything else to add, but that I agree with you.

I think you may have misunderstood the guidance from your legal team on this.

There is literally no way to prevent a candidate from disclosing a protected characteristic during an interview. Some obvious examples: they might show up to the interview in a wheelchair, they might be wearing a religious garment, they might be visibly pregnant, and so on...

What legal doesn't want, is you asking questions directly intended to elicit that kind information when the candidate didn't volunteer it. Asking the candidate a direct question like "are you planning to have kids?" makes it sound like that information will be used in the hiring decision.

I gave candidates the option to teach me something, if they wanted to.

As you can see from the examples I gave, what they taught me was completely up to them.

It was just a fun exercise that I and every candidate enjoyed.

Forgive me, but I just don't see the problem here.

If you did decide to teach you something about the Quran, that would be great!

I don't know enough about these holy books, and I always welcome a chance to learn more.

> Forgive me, but I just don't see the problem here.

There is no problem. But some people will create problems out of thin air, when there is none.

100% true.

Completely unrelated, but I find that other people will create problems by making an off-colour statement, something mildly offensive, or disrespectful and then when they get a reaction they say "why are you making a problem where none exists".

DARVO/gaslighting behaviour, and I wish it was rarer than it is.

This is outrage bait you have fallen prey to. Now you're spreading it. You're better than this.

Companies with armies of attack lawyers riding helicopter gunships don't make great targets for this kind of thing, in general. I shudder to think about trying to take Apple or Disney to court.

It's somewhat overblown - obviously anyone can try to submit a demand letter about anything. My experience with legal in the hiring process is they want to avoid obvious own goals, and document the process so that clear reasoning can be expressed. Then, unless you really do something obviously discriminatory, you can tell people who claim they've been discriminated against to go pound sand.

There are lots of good tactical reasons to settle claims like that, so lawyers may advise you to settle, but if you're of the "we don't negotiate" mindset, in my experience most lawyers are quite happy to gear up for a fight with the right groundwork in place.

What, what? Harmonicas are tunable? TIL...

Oh yes, they are!

You just need a small file, like a point file. Any gasheads remember those? And a single edge razor blade or the like to lift the reed.

To raise the pitch, you file the end of the reed, making it lighter so it vibrates faster.

To lower the pitch, you file near the attached end of the reed. I am not sure on the physics of this and would appreciate anyone's insight.

The specific tuning I've done many times is to convert a standard diatonic harp (the Richter tuning) to what is now called the Melody Maker tuning.

The Richter tuning was apparently designed for "campfire songs". You could just "blow, man, blow" and all the chords would sound OK.

Later, blues musicians discovered that you could emphasize the draw notes, the ones that are easy to bend to a flat note to get that bluesy sound. This is called "cross harp". For example, in a song in G you would use a C harp instead of one tuned in G.

The problem with cross harp is that the 7th is a minor 7th and you have no way to raise it up to a major 7th if that would fit your song. And the 2nd is completely missing! In fact you just have the tonic (G in this case) on both the draw and blow notes where you might hope to hit the 2nd (A). There is no A in this scale, only the G twice.

To imagine a song where this may be a problem, think of the first three notes of the Beatles song All My Loving. It starts with 3-2-1. Oops, I ain't got the 2. Just the 1 twice.

This is where the file comes in. You raise the blow 1st to a major 2nd. And you raise the minor 7th to a major 7th in both octaves.

Now you have a harp with that bluesy sound we all love, but in a major scale!

Re: lower pitch, I'd hazard a guess that you're basically reducing the restoring force, so the resonant frequency goes down. Think of the attachment point as a bunch of springs in parallel; you snip a few of them and the overall spring constant is reduced. Or another way to think of it: Imagine you had a reed of a given width and then added mass to the end by making it wider at the non-attached end. You'd expect the frequency to go down.

I'm sometimes tuning accordions, which is rather similar, but if you want to lower the tone by a lot, you can plop some solder close to the tip of the reed.

It's pretty clear why that's the opposite of filing off material close to the tip, so obviously the tone goes lower.

In my mental image, filing close to the base of the reed gives the reed a similar shape as putting extra material next to the tip (thinner at the base, thicker next to the tip), and that's why it behaves the same.

Aha! You taught me something today. ;-)

The solder on the tips reminds me of a doctor visit years ago. I thought I may have broken a finger, and when I got to the doctor's office I mentioned to the receptionist that I had a high-deductible (HSA-compatible) insurance plan.

The physician's assistant said, "We could send you over for an X-ray, but since you're paying out of pocket, we can start with a simple test." He pulled a contraption out of his desk drawer and asked, "Do you know what this is?"

I said, "Yeah, a tuning fork. And based on the size and those heavy weights on the tips, it must be tuned to a rather low frequency."

It looked like this one:

https://www.stethoscope.com/adc-tuning-fork-128hz-500128/

He said, "Yep. So we get it vibrating and then touch the base to your finger. If you have a fracture, it will hurt because of the broken bone ends jiggling against each other. Then we will go for the X-ray to get more details. If it doesn't hurt, you are good to go. Is that OK?"

"Sounds good to me!"

It didn't hurt at all, and I just had to pay for a simple office visit instead of an expensive X-ray.

Isn’t every physical thing tunable since its materials have a resonant frequency?

Sure. Seems so, right? I would agree.

Once we are there, it then is all about techniques. Working from first principles is going to get us there, but there are likely pitfalls, traps, all manner of gotchas laying in wait...

And there we now have a basis for further discussion.

Probably so. Or in some cases, multiple resonant frequencies all at once, like a bell. I asked my friend Miss Chatty (ChatGPT) about this and she had some interesting insights:

https://chatgpt.com/share/67a2aade-5870-8012-b211-1bd749a2d7...

This post is some unintentional satire how IBM operates (nobody invents anything in IBM anymore).

The opening question is a copy of what was done before (probably by someone who doenst work at IBM anymore) and all the new stuff is stolen from outsiders.

Seriously? I can assure you with 100% certainty that the most important point - "teach me something" - was entirely my own innovation.

I am tempted to take some offense at your comment. But I have to assume that you mean well.

You are correct that I don't work for IBM any more. What does that have to do with anything?

I'm surprised at the negativity your approach seems to have sparked in a few, but I found it really great, probably very effective as well and will probably start to use it at some point.

Thanks for sharing.

So was I! Frankly, having done similar things, I found their comment here lucid, well thought out and valuable enough to write some things in resonance.

My take is the negatives may just be people leaning hard on, "if it seems too good to be true...

Thank you! And please do feel free to use the idea, or change it and make it your own. The great thing is that it starts the interview with the candidate being the expert at something, and I am their student.

(3) is biasing the process strongly in favour of people who spin a good story. If you're looking for a certain team culture then OK but this is going to neatly screen out anyone who just wants to do their job well and doesn't particularly know how to sell the extra-curriculars they have.

Specifically for engineering I think it could work if you really press on that it's about teaching something. A core part in an engineering team is to walk eachother through concepts, so judging how you can explain concepts to someone is actually a good thing i believe?

People were free to teach something in programming.

That person can teach OP something about code then? They said it doesn't have to be work-related, but it can be.

I don't know that many programmer/developer jobs where you can just put your headphones on and code without ever talking to another person.

Being able to explain/teach your work is part of "doing the job well" for a developer (IMO).

Sure, they can do that. Then the average interviewer is going to recommend hiring the person who was the best at spinning a good story rather than the person who has the clearest explanation (which might easily come across as boring or simplistic). The problem here isn't getting through the interview; it is what the interviewer does next in the hiring pipeline.

How should you conduct the interview, then, if:

... Classic interviewing techniques of "explain how X algo works" or "write code to solve Y" will unfairly bias against interviewees that don't test well under pressure, but would otherwise be a good coworker.

... "Teach me something interesting to you" or "tell me stories about past experiences" will unfairly bias against interviewees that are shy, soft spoken, or are mildly socially awkward, but would otherwise be a good coworker.

Given the above awareness of where bias can emerge, how should the interview be done in order to get a candidate that knows what they're doing, and works well with the rest of the team?

Other comments mention relying more on recruiters and referrals, but that isn't always an option.

I don't see what useful signals are supposed to come out of an interview. If referrals and recruiters aren't an option I'd probably try to skip the interview altogether and go with a long probation period (3-6 months). Or possibly have a short 20 minute free-form interview talking about their last day job and expectations of the new one with a very short list of major red flags ("doesn't want the job", "unable to form sentences") then block candidates who raise them.

The long probation approach completely ignores the very real costs of onboarding someone new.

It takes time, money and people to bring someone in, and hiring is actually quite a risk for many companies (unless they're huge and/or in a hiring frenzy).

If a candidate doesn't work out, that's a lot of time and money down the drain, and potentially lost work, and disruption to teams and timelines, etc..

Most people don't get this part, and I think that's why they don't understand why interview processes are structured the way they are.

You really want to do the best possible evaluation, on all fronts, at the start. The longer a bad candidate stays in your pipeline or company, the more expensive and disruptive it gets.

- How do they take a business problem and model it into code - How do they debug their own code - Is their code easy to read - Do they name their variables/fields/methods/classes in easy to understand and consistent ways or are the names confusing or inaccurate - How do they take constructive criticism - How collaborative are they - Do they think about the problem first or do they just start hacking away - When asked to add a feature to existing code, do they start hacking or do they write out a test describing the new functionality first - When confronted with vague requirements, how well do they ask questions to get the information they need - How much experience do they have with algorithms, database design, systems design, building things so they scale well

If it were possible to work all that out in the interview then there wouldn't be any bad hires.

As a wishlist I like it, I just don't see how you're going to assess all that in an interview. You'll notice that the technique of the day ("teach me something") doesn't address any of the dot points and that holds for ... pretty much any technique. Interviews are a weak process for assessing anything.

Somehow this reminds me of my old physics professor, who often had simplistic, almost boring explanations of things, like jiggling atoms making friends with each other, or how a train stays on the track.

His name was Richard Feynman.

https://www.youtube.com/watch?v=nYg6jzotiAc

(Highly recommended video.)

I should also note that "teach me something" was intended to give the candidate a chance to be the expert at the beginning the interview.

It wasn't something they would be judged on, it was an ice-breaker. And a fun one, based on the feedback I got from candidates.

A big part of any engineering job is casual technical communication. This seems like it's entirely fair line of questioning. And the candidate gets to pick their strongest topic. What more do you want?

I don't think this approach favors the talkers / story spinners too much. It is the other way around!

It is a nerd test. You can get every nerd talking if you ask them to tell you about a special skill or project they really care about.

I remember quite a lot of these lunch conversations:

Me: Hey, what's up? Do you like the salad?

Introvert Nerd: Mmmm-Hmmm.

Me: Great weather outside!

Introvert Nerd: Mmmm-Hmmm.

Me: I was hiking last weekend in the mountains and got caught in a storm.

Introvert Nerd: Mmmm-Hmmm.

Me: Does your work project proceed well?

Introvert Nerd: Mmmm-Hmmm.

Me: I've heard that you regularly cook medieval dishes and you created a food medievality detector using a Raspberry Pi and a horseshoe?

Introvert Nerd: Oh, yes! You know, measuring the medievality of a dish is not as simple as it sounds! Obviously, there are no American foods like tomatoes or potatoes allowed, but did you ever think which spices were common in Europe in the High Middle Ages and why that changed in the Late Middle Ages... ...

The problem is in leaving the topic of what to teach open. To some people, this may feel like freedom. To others, in the context of an interview where the purpose is to judge the candidate, it will just lead to a bunch of stress from trying to guess which types of things to teach are currently a la mode on the interview circuit.

For a fairly casual question I'd see it more like an ice breaker or a way to build some rapport, but it's down to how you ask the question as well. Letting people know in advance (as the parent suggested) rather than dropping it on the fly is a good enough accommodation for people who would be anxious about that.

I find interviews quite tense and I generally dislike having them (on both sides of the table), but throwing in a few disarming questions here and there throughout really takes the edge off. A conversational, or more cerebral, approach like that can suit some people far better than live code tests or quick-fire Q&A.

Really nice!

I wish more bosses were like you.

Well thank you! I wasn't the boss, just a team member who helped out with interviews.

I am grateful that so many people in this thread found the ideas useful.

I immediately want to learn about all these cool things you listed.

I work as a developer and as an interviewer (both freelance). Now I want to integrate your point 3. into my interviews, but not to choose better candidates, just to learn new stuff I never thought about before.

It is your fault that I see now this risk in my professional life, coming at me. I could get addicted to "teach me something". 'Hey candidate, we have 90 minutes. Just forget about that programming nonsense and teach me cool stuff'

very thoughtful idea!

Kudos to you - this sounds like a fantastic interview format.

I especially like that you're prepping them very clearly in advance, giving them every opportunity to succeed, and clearly setting a tone of collaboration.

In person design and coding challenges are a pressure cooker, and not real-world. However, giving people the choice, seems like a great way to achieve the balance.

Honestly, I'm really just commenting here so that this shows up in my history, so I can borrow heavily from this format next time I need to interview! :) Thanks again for sharing.

"Teach me something" is how the test prep companies would interview instructors. Same idea—have a candidate explain a topic they are totally comfortable with—but they were more focused how engaging the lesson was, how the candidate handled questions they didn't expect, etc. moreso than I expect you would if you use this in a IC coding interview. It's a neat idea though, I can imagine lots of different ways it would be handy.

I spent a decade and change doing adult instruction in CAD. Early on, many were still transitioning off 2d hand drawings. Boy, was that an art! I got to do a few the very old school way and have serious respect for the people who can produce a complex assembly drawing that way. Same for many shapes that may feature compound curves, for example.

But I digress!

Asking them what they wanted or would teach me was very illuminating and frankly, fun all around! I was brought a variety of subjects and not a single one was dull!

Seems I had experiences similar to yours.

One of my questions was influenced by someone who I respected highly asking, "what books are on your shelf at home?"

This one almost always brought out something about the candidate I would have had no clue about otherwise. As time advanced, it became more about titles because fewer people maintain a physical book shelf!

I love this. I hope you don't mind, but I'm going to steal it for when I am back in the saddle interviewing folks. We need much more of this and a lot less of the adversarial BS.

Please do! If you ever want to talk about some of the details, you know where to find me.

Are you still interviewing?

I am! But I am now on the other side of the virtual table.

IBM conducted a mass layoff a few months ago, and as our little team lost our only customer at the time (this is public information), many of us were let go. Ah well.

One door closes, another opens.

If anyone out there is curious about "who is this guy with the strange and interesting ideas about interviewing", you can find my LinkedIn and other contact info in my HN profile.

As someone with a pretty long career already, and who's comfortable talking about it, I was a bit surprised that in three interviews last year nobody asked a single thing about any of my previous work. One was live coding and tech trivia, the other two were extensive take-home challenges.

To their credit, I think they would have hired "the old guy" if I'd aced their take-homes, but I was a bit rusty and not super thrilled about their problem areas anyway so we didn't get that far. And honestly it seems like a decent system for hiring well-paid cogs in your well-oiled machine for the short term.

Your method sounds like what we were trying to do ten years ago, and it worked pretty well until our pipeline dried up. I wish you, and your candidates, continued success with it: a little humanity goes a long way these days.

So, did you find a company that you are happy with (interviewing or otherwise)? I would be really interested to know how you are dealing with tech landscape changes lately, and your plans for staying in tech ...

Wishing you all the best for your career!

>>Do either no take home test, or one that takes at most two hours. I do discuss the solution candidates came up with, so as long as they can demonstrate they know what they did there, I don't care too much how they did it. If I do this part, it's just to establish some base line competency.

The biggest problem with the take home tests are not people who don't show up due to not being able to finish the assignment, But that those people who do, now expect to get hired.

95% people don't finish the assignment. 5% do. Teams think submitting the assignment with 100% feature set, unit test cases, onsite code review and onsite additional feature implementation still shouldn't mean a hire(If not anything, there are just not enough positions to fill). From the candidate's perspective, its pointless to spend a week working and doing every thing the hiring team asked for and still receive a 'no'.

I think if you are not ready to pay 2x - 5x market comp you shouldn't use take home assignments to hire people. There is too much work to do, and receive a reject at the end. Upping the comp absolutely makes sense as working a week to get a chance at earning 3x more or so makes sense.

We did a lot of these assignments and no one assumed that they will be hired if they complete it. Its about how you communicate your intent. I always told the candidates, that the goal of the task is 1. to see some code and if some really basic stuff is on point and 2. that you can argue with someone about his or her code.

>>We did a lot of these assignments and no one assumed that they will be hired if they complete it. Its about how you communicate your intent.

Be upfront that finishing the assignment doesn't guarantee a hire and very likely the very people you want to hire won't show up.

Please note that as much as you want good people to participate in your processes. Most good talent doesn't like to waste its time and effort. How would you feel if someone wasted your time and effort?

I am in germany, so by far not the same situation as in other areas of the world. If I would get such an assignment myself and I have the feeling that this will help the company and also me to verify, if it is a fit, I will do that 1 to 3 hour task very happily.

If I have a public portfolio of existing projects on GitHub, couldn't that replace an assignment? Choose one of my projects (or let me choose one), and let's discuss about it during the review interview.

If you're expecting a week's worth of work from them you'd better pay them for their time, if they turn up.

> From the candidate's perspective, its pointless to spend a week working and doing every thing the hiring team asked for and still receive a 'no'.

Uhhh yeah. That would really piss me off. Like reviewbombing glassdoor type of pissed.

Most of the time, those take home tests cannot be done in 2 hours. I remember one where I wasn't even done with the basic setup in 2 hours, installing various software/libraries and debugging issues with them.

People mostly fix some issue, or implement a feature at work. And then somebody outside can do the same in the same time.

In reality they need to look at this like onboarding a new team member.

How does that process handle people who have been out of work for a few years and can pass a take-home technical challenge (without a LLM) but cannot remember a convincing level of detail on the specifics of their past work? I’ve been experiencing your style of interview a lot and running up against this obstacle, even though I genuinely did the work I’m claiming to have done.

Especially people with ADHD don’t remember details as long as others, even though ADHD does not make someone a bad hire in this industry (and many successful tech workers have it).

I do prefer take-home challenges to live coding interviews right now, or at least a not-rushed live coding interview with some approximate advance warning of what to expect. That gives me time to refresh my rust (no programming language pun intended) or ramp up on whichever technologies are necessary for the challenge, and then to complete the challenge well even if taking more time than someone who is currently fresh with the perfectly matched skills might need. I want the ability to show what I can do on the job after onboarding, not what I can do while struggling with long-term unemployment, immigration, financial, and family health concerns. (My fault for marrying a foreigner and trying to legally bring her into the US, apparently.)

And, no, my life circumstances do not make it easy for me to follow the common suggestion of ramping up in my “spare time” without the pressures of a job or a specific interview task. That’s completely different from when I can do on the job or for a specific interview’s technical challenge.

I can't say I've interviewed someone to which this applies - unfortunately! Probably just doesn't get surfaced by my channels.

I would definitely not expect someone out of work for a while to have any meaningful side projects. I mean, if they do, that's cool, but I bet the sheer stress of not having a job kills a lot of creativity and productivity. Haven't been there, so I can only imagine.

For such a candidate, I'd probably want to offer them a time limited freelance gig rather quickly, so I can just see what they work like. For people who are already in a job, it's more important to ensure fit before they quit there, but your scenario opens the door to not putting that much pressure on the interview process.

Thanks for being understanding and compassionate about the situation! I like your idea of a time limited freelance gig, though immigration obstacles do sometimes make that quite complicated (as in my current situation). It's way better than not considering that people might have a reason for a resume with gaps or other irregularities which is not being a bad employee.

Beyond immigration types of obstacles, somehow recruiters and hiring managers rarely consider how many bad managers, bad executives, and bad companies there are when evaluating gaps and short tenures in an employee's resume. But equally, discussing those matters during an interview risks being seen as unprofessional and unreasonably negative. "Why did you leave this company?" "Oh, the warnings I got about the leadership from a former employee before I joined turned out to be true, and they didn't want to hear professionally presented necessary feedback about emotional safety in an emergency incident response situation, so they fired me without a single meaningful 1:1 discussion other than trying to assign blame." / "Oh, the CEO was enough of a problem that the investors eventually replaced him in a subsequent funding round despite him being the majority shareholder, but that was long after I had resigned or been fired due to that CEO's particular problems." / "Oh, they communicated in a very idiosyncratic way which didn't work for me and which I haven't seen at any company before or since." / etc. Most of that doesn't fly in an interview, but equally, even if it did, saying too much of that sounds like making up excuses for oneself even when it's 100% true. Same thing for why a job search doesn't succeed quickly.

Ours is a messy and imperfect industry, and it sucks that interview candidates have to pretend otherwise to seem like they'll be reasonable employees. Meanwhile the companies and executives that act in those ways get to present whatever positive and successful image they want, and they get to praise each other for firing fast or cutting costs with attrition or layoffs.

Well, it could be "just" naive hiring processes. It's pretty natural for people to use themselves and their immediate circles as thought guinea pigs for any process they come up with. And unfortunately, quite often, people in positions of power don't work their way up there, so it takes more effort than they are usually willing to invest to relate. Which is a shame, because doing things beyond what everybody else is doing can be a huge advantage.

One of the problems I see in big corporates is that feedback cycles for bad policy can be so long, whoever caused something is far away when the effects hit. AKA nobody really cares. But to be fair, I've always tried to care, and worked with many others who do.

This is slightly tangential to your questions, but to address the "remembering details about your past work", I've long-encouraged the developers I mentor to keep a log/doc/diary of their work.

If nothing else, it's a useful tool when doing reviews with your manager, or when you're trying to evaluate your personal growth.

It comes in really handy when you're interviewing or looking for a new job.

Julia Evans calls it a "brag doc": https://jvns.ca/blog/brag-documents/

It doesn't have to be super detailed either. I tell people to write 50% about what the work was, and 50% about what the purpose/value of the work was.. That tends to be a good mix of details and context.

Writing in it once a month, or once a sprint, is enough. Even if it's just a paragraph or two..

one of my buddies was a navy psychologist assistant. was a corpsman (navy medic) who moved that way.

dude maintained a "me wall" of achievements, commendation medals, etc. plus a file full of other crap. he mentioned he got a lot of promotions because you have a section to fill out where you describe all the shit you did over the least year, and he always came with examples...

Yeah, my resume does include a summary of what I did each job, but it sounds like the brag doc idea would involves much more detail.

Honestly, putting those details in writing in a way that I retrain after leaving might leave me vulnerable to claims of violating my corporate NDA. But legalities aside, yes, it would help with these kinds of interview issues - prospectively only of course, not retrospectively.

Annoyingly, it's also exactly the kind of doc that's difficult for people with ADHD to create and maintain rigorously, even though people with ADHD are more likely than average to need the reminders of those details. A lot of ADHD problems are that kind of frustrating catch-22, especially when interacting with a world that is largely designed around more typical brain types.

Got it.. I will admit to being unfamiliar with the challenges of ADHD so I apologize if my suggestion appeared to disregard said challenges.

If I can ask a follow-up question: Is it the act of writing it down, or the whole recall aspect of it that is challenging? Or something else?

I've personally starting using audio transcription with a voice recorder to capture ideas more easily when I'm not at my computer (I hate typing on my phone keyboard) and that has made building these kinds of diary/log/notes documents a lot easier because I can just speak my ideas into the recorder whenever it is convenient for me, instead of waiting until I'm at a computer.

Just a suggestion, but I have done 3-5 projects a year for a long, long time, and as an executive (for almost a decade) had dozens of projects annually I was overseeing and/or contributing to. I am not a fan of LinkedIn, but I did eventually start logging at least some of the projects I do in their "projects" section. That helps me remember and revisit some of the projects, and when you go back 10 years later it's sort of joyful walk down memory lane sometimes.

I feel like take home tests are meaningless and I always have. Even more so now with LLMs, though 9/10 times you can tell if it's an LLM-people don't normally put trivial comments in the code such as

> // This line prevents X from happening

I've seen a number of those. The issue here is that you've already wasted a lot of time with a candidate.

So being firmly against take home tests or even leetcode, I think the only viable option is a face to face interview with a mixture of general CS questions(i.e. what is a hashmap, benefits and drawbacks, what is a readers-writer lock, etc) and some domain specific questions: "You have X scenario(insert details here), which causes a race condition, how do you solve it."

    > I feel like take home tests are meaningless and I always have. Even more so now with LLMs

This has been discussed many times already here. You need to set an "LLM trap" (like an SSH honey trap) by asking the candidate to explain the code they wrote. Also, you can wait until the code review to ask them how they would unit test the code. Most cheaters will fall apart in the first 60 seconds. It is such an obvious tell. And if they used an LLM, but they can very well explain the code, well, then, they will be a good programmer on your team, where an LLM is simply one more tool in their arsenal.

I am starting to think that we need two types of technical interview questions: Old school (no LLMs allowed) vs new school (LLMs strongly encouraged). Someone under 25 (30?) is probably already making great use of LLMs to teach themselves new things about programming. This reminds me of when young people (late 2000s/early 2010s) began to move away from "O'Reilly-class" (heh, like a naval destroyer class) 500 page printed technical books to reading technical blogs. At first, I was suspicious -- essentially, I was gatekeeping on the blog writers. Over time, I came to appreciate that technical learning was changing. I see the same with LLMs. And don't worry about the shitty programmers who try to skate by only using LLMs. Their true colours will show very quickly.

Can I ask a dumb question? What are some drawbacks of using a hash map? Honestly, I am nearly neck-bearded at this point, and I would be surprised by this question in an interview. Mostly, people ask how do they work (impl details, etc.) and what are some benefits over using linear (non-binary) search in an array.

If you really need to test them / check that they haven't used an LLM or hired someone else to do it for them (which was how people "cheated" on take-home tests before), ask them to implement a feature live; it's their code, it should be straightforward if they wrote it themselves.

This is an interesting point. During the code review, you can add a small twist to the problem, then ask them how they would modify the code.

"Drawbacks" was the wrong word to use here, "potential problems" is what I meant - collisions. Normally a follow up question: how do you solve those. But drawbacks too: memory usage - us developers are pretty used to having astronomical amounts of computational resources at our disposals but more often than not, people don't work on workstations with 246gb of ram.

I think the better word is tradeoff since there are no perfect data structures for each job. The hasmap has the advantage of O(1) access time but the drawback of memory usage, an unsorted nature and the depends on a good hashing function to minimize collisions. A vector is also O(1), but it has an upfront memory cost that cannot be avoided. A map has a O(Log(n)) access cost, but has less memory usage, is sorted by nature and the comparison function is easier to implement. Three similar data structures, but each with its own tradeoffs.

Good point about collisions. When I wrote the original post, I didn't think about that. As a primarily CRUD developer, I never think about collisions. The default general purpose hash map in all of my languages is fine. That said: It does matter, and it is a reasonable topic for an interview!

What if I use an LLM but I understand the code?

The drawback is that elements in a hashmap can’t be sorted and accessing a specific element by key is slower then accessing something in an array by index.

Linear search is easier to implement.

These are all trivial questions you ask to determine if a person can develop code. The hard questions are whether the person is the cream of the crop. The amount of supply of developers is so high most people don’t ask trivial questions like that.

    > What if I use an LLM but I understand the code?

That's OK. I wrote: <<And if they used an LLM, but they can very well explain the code, well, then, they will be a good programmer on your team, where an LLM is simply one more tool in their arsenal.>> If anything, I would love it if someone told me that they used an LLM and explained what was good and bad about the experience. Or maybe they used it and the code was sub-par, so they need to make minor (or major) changes. Regardless, I think we are kidding ourselves if people will not make (prudent and imprudent!) use of LLMs. We need to adapt.

If you are evaluating how well people code without LLMs you are likely filtering for the wrong people and you are way behind the times.

For most companies, the better strategy would be to explicitly LET them use LLMs and see whether they can accomplish 10X what a coder 3 years ago could accomplish, in the same time. If they accomplish only 1X, that's a bad sign that they haven't learned anything in 3 years about how to work faster with new power tools.

A good analogy of 5 years ago would be forcing candidates to write in assembly instead of whatever higher level language you actually use in your work. Sure, interview for assembly if that's what you use, but 95% of companies don't need to touch assembly language.

> ... LET them use LLMs and see whether they can accomplish 10X what a coder 3 years ago could accomplish...

Do you seriously expect a 10x improvement with the use of LLMs vs no LLMs? Have you seen this personally, are you 1 10th the developer without an LLM? Or is the coding interview questions you ask or get asked, how to implement quicksort, or something?

Let's make it concrete, do you feel like you could implement a correct concurrent http server in 1/10th the time with an LLM than what you could do it without? Because if you jut let the LLM do the work I could probably find some issue in that code or alternatively completely stump you with an architectural question unless you are already familiar with it, and you should not be having an LLM implement something you couldn't have written yourself.

> implement a correct concurrent http server in 1/10th the time with an LLM than what you could do it without

Absolutely fucking yes.

In that case, could you begin proving that point by having it write an http request parser. Let's make it easy and have it require content length header and no support for chunked encoding at first. Csn pick any language you like but since thats such critical infrastructure it must export a C API. Let's also restrict it to HTTP 1/1.1 for the sake of time.

Concidering this would probably at most take a days work to get at least a workable prototype done if not a full implementation, using an AI you should be able to do it in a lunch break.

To add: I can very well imagine this process isn't suitable for FAANG, so I can understand their university exam style approach to a degree. It's easy to arm chair criticise, but I don't know if I could come up with something better at their scale. These days, I'm mostly engaged by startups to help them build an initial team, I acknowledge that's different from what a lot of other folks hire for.

Is your job consulting for hiring technical positions?

I have a consultancy - we mostly do development, but I also do quite a bit of "consulting CTO" work. Quite often, that means helping with hiring.

But hiring is nowhere near my full time job, thankfully :D

That’s cool. My main job is being a research scientist but i also work at a computational science consultancy that essentially builds numerical simulations for hire along with other software consultancy jobs. I occasionally have been asked to serve as an outside consultant for hiring data science positions. I’ve been wondering how to grow that a bit into more regular work because I enjoy it. Any thoughts on that?

Well, my contracts pretty much all came through my network, someone recommending me, and then clients recommending me, and so on. Not sure if I just got lucky or if this actually works, but my first step would be to tell some relevant folks you know that you're doing this now, see if they have any advice or know someone who might be interested. Another approach would be to ask the consultancy you work with if they want to add this service you can provide to their portfolio, see if it's something they could sell to existing or new clients.

However, these days, people seem barely OK with paying for a recruiter. They think just because more people than usually are looking, they get to just lean back and great candidates show up.

IMHO, if anything, it got more difficult to hire. More noise to work through, people in existing roles are more reluctant to switch, candidates hustle more because they need a job etc. I absolutely think it's a useful service. But I don't know if it's easy to market.

Why not? Plenty of large organizations hire this way. My first employer is bigger than any FAANG company by head count, and they hired this way. Why is big tech different?

The desire for a scalable, standardized scoring mechanism so they can avoid lawsuits.

Why would a big-non-tech company not have the same desire to avoid lawsuits?

And yet this interviewing problem seems to only affect tech companies.

A lawsuit on what basis?

Well, I respect the scale and speed. My process was still working fine at ~5 per month. I have doubts it'd work with orders of magnitude more. There's a lot of intuition and finesse in there, that is probably difficult to blindly delegate. Plus, big companies have very strong internal forces to eliminate intuition in favour of repeatable, measurable processes.

When you're operating a small furniture company, your master craftsmen can hand-fit every part. Match up the hole that's slightly too big with the dowel that's slightly too big, which they put to one side the other day.

When you're operating a huge furniture company, you want every dowel and every hole the same size. Then any fool can assemble it, and you don't have to waste time and space on a collection of slightly-too-large dowels.

To scale up often means focusing on consistency and precision, and less on expert judgement and care.

Also human judgement is loaded with bias. Height influences outcome and this is a statistically proven phenomena. Ask any interviewer in this thread and they will claim height and good looks aren’t a factor but the statistics prove them wrong.

The literal only way to get rid of that is quantitative tests.

That's a nice story, and completely irrelevant. Different orgs scale different ways, and ultimately hiring is done at the team level. You have a vacancy or need on a team to fill. Big org or small org, the situation is the same.

Interestingly large companies will often have a hiring pipeline that isn't specific to a single team and role.

In the past Google made a lot of hires where they first give people a phone screen, then five in-person whiteboard interviews, then the interviewers send a dossier to a committee, then that committee decides whether to hire or not, then "team matching" lets hiring managers see the resumes.

So the interviews are basically conducted by random people, who have no idea what team the candidate will end up on.

Of course if you look at the number of employees Google has, and the average tenure, you can see why they make hires like Ikea makes chairs.

Yes that is how FAANG works, but they’re the oddity. Most large organizations don’t hire that way. Google invented this method of hiring, and others copied. It is debatable whether it has been good or not for them.

Usually what happens in a large org is (1) a department gets allocated a head count, and it eventually trickles down to team allocations; (2) a team needs a role that they can’t backfill from the rest of the company; (3) the team lead puts out an ad, receives resumes (directly or through HR), schedules interviews, and makes the hiring decision themselves. Exactly the same as a small company or startup in the last step.

Was it in the US? Unless it’s Walmart, there is no company in the US that is larger than the largest FAANG by headcount.

    > Put the candidate at ease - nervous people don't interview well

This is great advice. I have great success with it. I give the same 60 second speech at the start of each interview. I tell candidates that I realise that tech interviews are stressful -- "In 202X, the 'tech universe' is infinitely wide and deep. We can always find something that you don't know. If you don't have experience in a topic that we raise, let us know. We will move to a new topic. All, yes, all people that we interviewed had at least one topic where they had no experience, or none recent." Also, it helps to do "interview ramp-up", where you start with some very quick wins to build up confidence with the candidate. It is OK to tell them "I will push a bit harder here" so they know you are not being a jerk... only trying to dig deeper on their knowledge.

Putting candidate at ease is definitely important.

Another reason:

If you're only say one of four interviewers, and you're maybe not the last interviewer, you really want the candidate to come out of your interview feeling like they did well or at least ok enough, so that they don't get tilted for the next interview. Because even if they did really poorly in your interview, maybe it's a fluke and they won't fail the rest of the loop.

Which is then a really difficult skill as an interviewer - how do you make sure someone thinks they do well even if they do very poorly? Ain't easy if there's any technical guts in the interview.

I sure as shit didn't get any good at that until I'd conducted like 100+ interviews, but maybe I'm just a slow learner haha

I’ve done the “at home” test for ML recently for a small AI consulting firm. It's a nice approach and got me to the next round, but the way the company evaluated it was to go through the questions and ask "fundamental ML bingo" questions. I don't think I had a single discussion about the company in the entire interview process. I was told up front "we probably won't get to the third question because it will take time to discuss theory for the first two".

If you're a company that does this, please dog food your problems and make sure the interview makes the effort feel valued. It also smells weird if you claim it's representative of a typical engineering discussion. We all know that consultancy is wrangling data, bad data and really bad data. If you're arguing over what optimiser we're choosing I'd say there's better ways to waste your customer's money.

On the other hand I like leetcode interviews. They're a nice equalizer and I do think getting good at them improves your coding skill. The point is to not ask ludicrously obscure hard problems that need tricks. I like the screen share + remote IDE. We used Code which was nice and they even had tests integrated so there wasn't the whiteboard pressure to get everything right in your head. You also know instantly if your solution works and it's a nice confidence if you get it first try, plus you can see how candidates would actually debug, etc.

Wow, that was a great write up. Can I interview with you? Lol, everything you wrote was really spot on with my own interview experiences. I tend to get super nervous during interviews and have choked up on many interviews asking for live coding on crazy algorithm problems. It state of hiring seems to be really bad right now. But I'll take your advice and try to get in contact with some recruiters

I would never show an interviewer how I code. Let alone allow them to give me 2 hours to solve a problem. It's contradicting that you know that developers "mostly shine when not under pressure", yet you set 2 hours to solve a problem (Sounds like you don't want to pay them for the take home). The positive feedback you're getting for your flawed process just shows how worse other, more common, recruiters are.

> 2. Do either no take home test, or one that takes at most two hours. I do discuss the solution candidates came up with, so as long as they can demonstrate they know what they did there, I don't care too much how they did it. If I do this part, it's just to establish some base line competency.

You need to do this to establish some baseline competency... for junior hires with no track record?

"Use recruiters and network: "

Recruiters have been notoriously bad in my experience. Relying on network has the potential to create bias and avoid good candidates that simply don't have an "in".

If they pass step 2, the first technical interview is to extend the solution to add/modify features.

Are you hiring now?

Reply on news.ycombinator.com