The Coffee Machine Hiring Test | Hacker News Reader

The Coffee Machine Hiring Test

manuel.darcemont.fr

6 pointsmegalomanu51 comments

These sorts of “gotcha” tests are silly. People are complicated, with many different personalities and behaviors. I can think of many reasons an excellent engineer might choose not to fix a broken coffee machine, especially in the stress of an interview.

If you want to know how somebody will work on the job, ask them to demonstrate the skills used on the actual job. This is called a “work sample interview.” (Leetcode problems don’t count, because most jobs don’t look like that. Neither do whiteboard interviews, for the same reason.)

If you want to know how somebody will behave in various situations, ask them to describe concrete examples of how they’ve reacted to those situations. Listen for deflection to hypotheticals and redirect towards lived experience. This is called a “behavioral interview.”

Don’t use tricks and gotchas.

This test is even more nonsense than that, the idea isn't even to have them try and fix it, the idea is to think in your head what you think they would do.

Yup, exactly! I’m not even convinced this person is the kind that would stop on the side of the road when they see someone with a broken down car. Are they the type that would try to see if they could fix it? If not, why not? Why would it be different from this interview, on the flip side why would one expect all travelers to be good samaritans?

It’s great if this is this person’s personality, but not everyone is like this.

> This is called a “work sample interview.” (Leetcode problems don’t count, because most jobs don’t look like that. Neither do whiteboard interviews, for the same reason.)

They do count, job sample tests doesn't need to be exactly like the job just similar. There is correlation between performance on those tests with performance on the job, far from perfect but its much better than the zero correlation you get from these cultural fit tests or free form discussions.

They're good for seeing how someone approaches a problem. Although with Leetcode everyone is just testing to see if you remember the One Simple Trick for that case instead of being able to talk through it.

There was found a reverse correlation (on a small sample at Google) the better you are at leetcode-like problems, the worse you are at the job.

The correlation is gone on a larger sample https://news.ycombinator.com/item?id=25426329

Yeah this test is terrible honestly. I fix all my own appliances and vehicles including complicated and time consuming repairs and at the office I'd be the first in practice to fix the coffee machine myself too if I thought I could.

But in an interview is almost certainly say something like "after checking for a few obvious things related to normal operation of the machine per the owners manual, I'd inform and defer to my supervisor." Or something like that.

Because anything else is a liability. I know that in fact, no corporate manager wants to go on record authorizing someone who isn't a certified repair technician to attempt repairing an appliance. Some may not want me as an expensive guy wasting my expensive time fixing something that a cheap guy can. It wouldn't take long before the correct aanswer is: order a new one.

Now you could nitpick that how I'd act in actuality doesn't match what I'd say I'd do, and in some contexts that's fair, but most of the time, an interview is about saying the "correct" answers even when you know that's not how the world actually works.

That being said, there is ONE trait I think is a good universal gotcha if you can find a way to test it. To borrow a term from the Army: is the person a BF? ("Buddy Fucker"). I e. The same person who will use the last of the coffee grounds without telling anyone or replacing them is the same person that will leave your team hanging in other contexts. It's a personality thing and it leaks out all over the place. Not that that's the best example, but the type of person to be incredibly inconsiderate to the person that comes behind them is not the guy you want on your team messing up your codebase.

Is that a LinkedIn post? It's wordy, doesn't explain anything, and tries to be a simple remedy to a complex problem. A problem irrelevant to the task to boot.

These tests are often done by people with a high image of the charge. Despite all the research showing futility, they are used in startups time and time again. They do go away with experience.

Anyone who spends time fixing a coffee machine is not earning their extremely high staff engineer salary.

My time is not suited to fixing coffee machines. People paid literally 1/6th what I am are tasked for that.

It would be good to message the person in charge of fixing or replacing it. All too often, others assume someone else already contacted them. Then, prepare a cup of tea and return to work.

Is this the same reason you wouldn’t pick up trash on the floor?

Trash on the flor has a known quick solution, a broken coffee machine doesn't and the fix often needs to be coordinated or you easily call in multiple people to repair it etc.

In general if a task needs to be coordinated you shouldn't try to do it yourself, at best you should notify a coordinator but usually they are already aware of it and you are just spamming them.

What kind of weird fantasy is this?

Picking up trash from the floor does not require special tools, and does not void the warranty of the floor.

From TFA : “if the office coffee machine broke down, how would the candidate react? Would they try to fix it, or would they wait for the office manager or the founder to do it instead?”

I guess we know what kind of candidate you would be :)

The engineer in me would at least try to characterize the problem. It might be something simple! And to flip your argument around, if it’s something you can fix in a minute, it’s going to be more expensive to call someone else to do it.

Typically when you buy an expensive percolator it comes with a service and supplies contract that gives you callout times and weekly deliveries. Why would you risk breaking something that cost thousands of dollars and risk voiding that contract?

Even something as “simple” as a commercial percolator has a lot behind it that we might not know about. Unless it’s a water line issue I wouldn’t touch it.

I don't trust most software engineers around physical machines.

As a software developer, I don’t even trust my own code. That’s why I rely on unit tests, integration tests, specifications, and 20 people who have never programmed in their whole life.

I still don’t see how it’s related to coffee.

> The engineer in me would at least try to characterize the problem. It might be something simple! And to flip your argument around, if it’s something you can fix in a minute, it’s going to be more expensive to call someone else to do it.

The engineer in me would report it to the office manager so they can deal with getting the right people to do it.

We have fancy/expensive/complicated superautomatic coffee machines at work. I know nothing about how those things work. I also don't know if that machine is even owned by the company or if it's leased. Do we have a service contract? Are we even allowed to touch it when it breaks?

Now, if it was my personal machine and it was out of warranty, sure I'll have a look. But knowing nothing about this machine my instinct would not be to tinker with it, but at the very least first get the answers to the questions above. I'm sure we all have experience with well-meaning laypeople trying to 'fix' a problem and making it worse. I don't want to be that person.

> I'm sure we all have experience with well-meaning laypeople trying to 'fix' a problem and making it worse. I don't want to be that person.

So much this. I feel like the article is selecting for the d-k peak of mt stupid; unearned confidence without a deep understanding of the situation.

> I know nothing about how those things work. I also don't know if that machine is even owned by the company or if it's leased.

Engineering is also about knowing stuff. If you don’t know these things about the appliances you use every day as part of your work routine, and it seems you can’t be bothered to even find out, that also speaks to what kind of candidate you’d be.

> If you don’t know these things about the appliances you use every day as part of your work routine

TIL the coffee machine is an essential part of my 'work routine'.

The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over, but it can’t, not without your help. But you’re not helping. Why is that?

Same shit.

It is a "cultural fit" test, a cousin of "how many golf balls can you put in a manhole cover" or whatever. The question aims to figure out if you're going to take on certain tasks, and how you would take them on; especially in the frame of something outside your vocation, and if you're a "doer" or not. He states this frame explicitly in the post. Do you tackle the problem, and how, or do you throw your hands up, stating that it's someone else's problem?

You definitely will not get hired by that guy: you misunderstood the question.

I wouldn’t want to work for a company who has coffee problems every time they hire someone, and can’t handle their own problems.

Or was it a stupid trap like the INTP psychological scam? How is it related to my future job?

It’s true in that there is comparative advantage in not doing it.

That’s what makes it a cultural scissor, and therefore a useful discriminator: there is no right answer, but there is definitely one kind of person these guys are selecting for.

I also like working with the kind of person who sometimes eschews comparative advantage. More than a decade ago, a startup I was at had our conference.

We ran barebones: front desk, ushers, lighting, everything. Run by members of the team. I was on engineering, at the front desk. It was great fun. Our customers loved it.

I thought I'd more or less seen most MBA nonsense around these days, but oh boy this is something else entirely. In order to assess a candidate, you're gonna think in your head "what do I think this person would do if the coffee machine broke down". You base your hiring decision for a person you just met and know more or less nothing about on what you made up in your head in a complete shower thought scenario.

I was worried at first that they used a broken coffee machine and had candidates make their own cup of coffee, and observed "how they handled the situation". If and only if they broke out a set of repair tools did they get invited back for a second round.

The imginary method outlined in the article might actually be worse.

They really could save a lot of time and effort by laying all the CVs out on the table and getting a Dowsing stick and using it to find the best candidate.

I’m not sure we can expect complete strangers who are not yet part of an organization to feel certain ownership that is necessary to take it upon themselves to try to fix something.

If you make it worse, for example, are they going to charge you for breaking it?

It’s kind of like when a kid, you don’t go into your friend’s house’s fridge unless you’re invited to open it, at least the first time visiting.

I have literally fixed a coffee grinder at one of the startups I worked at.

I think I would fail this test since most coffee machines are hard to repair unless it is something trivial and the idea of just repairing something is kind of odd. A better answer might be you work with the office manager, go to the store and grab a temporary one.

I also think this is a terrible idea for other reasons:

1. You are working at a small company and its easy to communicate, its broken and you are going to fix it. However, focus is critical even if it doesn't feel like it all the time.

2. It is hard to tell but it seems like saying you would talk to the office manager would cause you to fail but this seems like the most rational answer. You should work with them and see if you can fix it together or at least keep them in the loop.

Bias for action is great but you also need people who can focus, know what they are good at, and communicate with others.

edit: formatting (need to not post while raging)

Sure, fix the coffee machine with one hand while completing the leet code questions with the other hand in the limited time available for the interview.

Interviewer: Candidate seemed distracted and jittery.

I look forward to the next couple posts in this series: The Leaky Faucet Test and The Clogged Toilet Test.

I was hoping for the Office Fire Test.

This sounds like some stupid dating test. "I'll only marry a man who takes initiative so I'll break something a see if he fixes it."

I don’t drink coffee. Did I pass?

Seriously though, it’s not a good question since it depends not only on the person, but also on the organisation. Sometimes the right response is to try and fix it, sometimes to report it to office management, and even in some organisations they’ve been so on the ball there would be no reason to even report it because they’d havea repair organised already.

"if the office coffee machine broke down, how would the candidate react?"

I don't drink coffee, so I would have no idea it was broken in the first place.

But I think the correct answer to this is to check if something obvious is broken, and if you can't determine the issue within about five minutes, tell HR that it is broken. They'll know what to do.

What happens if I break your beloved coffee machine in the process? If I'm the new guy I'm not keen to find out.

I'd imagine that in almost every case that a candidate "passed" this test, they would eventually find themselves in a situation where a manager got control-freaky over them doing exactly that, and instead wanted the employee to do something else in the situation.

Enthusiasm is traditionally the straightforward signal to capture this.

I get that it’s intangible and could be faked to some degree but it also sometimes comes off so clearly and always leads to good outcomes in my experience.

Broken coffee machine? That’s a SEV0. Declare an incident.

Did you try rebooting it?

Remove the power cable, count to 5 and plug it in again.

Trying to service, or even just open up, any appliance at a workplace is against the Occupational Safety and Health regulations in my country.

I would just use the instant coffee that I keep in my desk drawer that I keep in case the machine breaks or the machine isn't stocked.

Why don't we cut to the chase and just start hiring based on star sign? What a joke

Read this, and now I'm making a coffee in my working machine..

> Why not start by observing how they would react to a broken coffee machine?

Because you’re not hiring someone to fix your coffee machine which can’t be fixed without knowing the insides of the specific model of that specific brand.

I would offer to buy instant coffee at the local supermarket. What kind of bullshit trap is this? Do you really want to hire people by confusing and scaring them? It’s on the same level as the LinkedIn crap I see once a week.

> Do you really want to hire people by confusing and scaring them?

A surprising number of startups do give me a frat house vibe, so . . . maybe?

I really don't like these kind of silver-bullet "hey look I easily solved a complex problem" self-congratulatory posts.

This feels like one of those things that should be openly explained as part of the culture instead of "guess our opinion". Anything else seems like reading the tea leaves that day to make hiring decisions.

Anecdotally, I've had two opposite experiences just like this. In the first I was walking and talking with the CEO about something important our team was working on when we came to a bunch of water on the floor (enough that clearly there was a problem like a broken pipe or fridge issue in the next room). He stepped over it but I stopped, completely distracted, and said shouldn't we do something about this? His response was short but poignant - he said we don't know the situation, we have great people that he trusts whose domain IS handling that kind of thing, and that there is only so much time in the day and that we, the CEO and a VP, needed to intentionally focus on big things only we could effectively handle even if that meant water on the floor for a little while longer.

I've also had the opposite experience where someone took it upon themselves to redesign one of the flows in our product because they (correctly) didn't think it served customers well. This person put a ton of good faith effort into this and produced a decent looking wireframe for a ux that didn't correctly understand how things work or what our goals for the project were. In fact this already had a good redesign that existed and was on the roadmap but he apparently hadn't been told about it. When he showed his design to people (directly to engineers asking them to build it instead of their actual tasks...) it caused a bunch of awkwardness for him and others. There was more drama after his manager continued to handle it poorly but all of it could have been avoided by more communication in any direction before deciding to just go do something outside of your normal zone.

These are just anecdotes but they highlight how I feel about this; I think "testing" people on how they respond to situations like this is both broken and disrespectful. The right amount of "just go fix it" is extremely subjective and depends on the culture of the company around you, how specialized the task that needs doing is, and the priority of what you could/should be doing instead.

To be clear I think a "doer" attitude is very valuable; I just don't think this is a fair or high signal way to evaluate it.

So the CEO that stepped over the water didn't even tell anyone about the problem? That just seems wrong. Would this person call the cops if he saw burglars breaking into a neighbor's house? Some things are everyone's job to report even if someone else will clean it up.

Did anyone at the company with the guerilla design effort realize that there was a huge communication problem or did they just spank the guerillas?

In the first one it was a very small office so it was taken care of immediately.

In the second one that's exactly what I took away from it - I escalated immediately, saying this was a concrete example of two separate communication failures. It didn't turn out particularly well and I wish it had been handled better (it was lost in a culture war between two execs, which was the cause of the original communication gap in the first place) but I feel like I did my part and no gorillas were spanked, just left feeling upset.

Reply on news.ycombinator.com