Hacker News Reader

Comment by stakkur | Hacker News Reader

The real truth is: hiring is a crapshoot. For any position and any field. Software engineering is no different.

There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire. If there were, everyone would be using it.

Instead, third-party HR monoliths have moved in and snatched up (and more or less created) the 'market' for screening and filtering applicants. Larger companies sigh, throw up their hands, and say 'well how else can we deal with hundreds of applicants?'.

As if that 'funnel' of applicants contains what you think you want. As if it's just an exercise in reductionism, each applicant a data point to evaluate.

Given this, why not hire lightly and fire lightly?

View on HN

Construction and trades takes that approach. It’s a mixed bag. I would say implicit job insecurity removes a lot of potentially good employees from the applicant pool. People who value stability and support families tend to be reliable, and some of them won’t work for someone who has a reputation for firing everyone as soon as things get slow or if they have an off day.

I think my industry (civil engineers) is a bit easier to hire for because credentialing weeds out people who are completely out of their depth, and lying about qualifications can get you in real trouble. Managers can assume you’re basically competent and can focus on hiring people with decent personality and a good reputation. Some of my software friends use brain-teasers to screen candidates, and it comes off as a bit silly to me.

Funny you should mention that. At my college (Cal Poly) I had fraternity brothers who were Civil Engineering majors. They would look up to me because I had a natural ease with computers. But in the end, I watched them sweat their way through classes that would have wasted me.

Many engineering fields are ones were being studious and knowledgeable can give you a solid career. Software development has a lot more upside potential for the flexibly creative and the lucky.

I feel this is less and less true for Frontend Development which is constantly evolving and you need to put in some serious effort to stay up to date.

This the engineering corollary of "The grass is always greener". For engineers it's "The other engineering disciplines are always harder"!

Which is another way of saying that the "advanced" subjects (vector calculus, multivariable calculus, fourier transforms) really aren't that bad if you're prepared to put the work in.

Probably because most people choose to study what comes easy to them.

I think the key is finding an intuitive way to grasp the subject matter, and then it doesn't feel like such a grind.

If material like this had been available 20 years ago it would have made a huge difference.

https://youtu.be/spUNpyF58BY

If you can do both, I highly recommend it. Programming abilities can multiply your productivity in another field several times.

Pretty much the case for every non-Software Engineering discipline.

But then again, people in the tech field will vehemently defend the system - because it's one of the very, very few fields where you don't need a college degree, can go on and "grind leetcode" for months, and land a six-figure job.

And it is because of that, that tech companies are making their recruitment process bordering the absurd. They're simply deadly afraid to vet frauds, that have "gamed" the system.

So in short - tech companies would rather let 10 decent candidates slip by, if it means they can block 1 fraud.

I've been to many engineering interviews, outside tech, and it is not even remotely the same process.

That fear may be true for candidates without experience and/or university degree. But why is it still present for candidates with lots of experience and education? In other words, if other technical/engineering disciplines can manage without this absurd hiring process, then why can't IT field (leaving aside the candidates without experience/higher education)?

Because every engineer has some outlier horror story in the lines of "We hired this dev/engineer with legit CV, but he turned out to be useless. Couldn't code a line of code"

I don't understand why they can't just fire that guy.

Universities are terrible at teaching programming/software engineering. Many also suck at teaching computer science. There's no incentives in academia to be good at teaching, and many lecturers only care about their research.

There are arguments here and on the blog post that the industry doesn't have credentials. We do have credentials, it's the CS degree, but universities suck at issuing them so we don't rely on them much. Sometimes not at all.

The people saying, hey, why can't we be like civil engineering are thus perhaps not sounding as re-assuring as they hope: why would universities be more trustworthy at teaching civil engineering than other kinds of engineering? The problems are incentive related. Ultimately the software industry is highly egalitarian and a bad university experience can't hurt your career permanently, like it could in most other fields.

When I say “engineer” in disciplines other than software, I mean somebody with a professional license. You can only get that by passing exams and having a combination of experience, education and recommendations to pass a state board. It’s much harder and riskier to fake that than to make up experience on a resume or make a blog to appear like a subject matter expert. Unless you know the candidate personally, it can take a lot of effort to uncover a carefully crafted lie. Plus, recognizing quality work in software isn’t even a solved problem, to my understanding.

Other engineering disciplines hire engineers who don’t have a PE all the time. There are more engineering disciplines where taking the PE exam is the minority than the other way around.

Of course they hire non-PEs. You can’t get a license without years (at least 4) of experience first. Most consultancies I know of would not hire anyone who couldn’t pass the FE, though.

My point is that there are plenty of electrical engineers, computer engineers etc... who never take the FE or PE exam.

> Given this, why not hire lightly and fire lightly?

Because any company that adopts such an approach will never be able to recruit anybody who wants to work in a stable environment with a consistent set of smart people who aren't in constant fear that they will be fired the next day.

Yeah, exactly... depends a lot on who is doing the firing and why too.

I suppose if ALL companies did that then there might be some sort of fluid equilibrium state where most companies had some mix of mostly competent engineers who chose to work there for positive individual choice/reasons. But in a more winner-take-all world where the "best" need to hire the "best", because that means second tier companies get the rejects and therefore will fail to compete and the chosen few get success and riches? Set the bar incredibly high and forget fairness. (also an extreme for purpose of conversation, just on the other end of the spectrum)

When you know you have a job for the next n years at least, you also tend to put more effort into efficient processes. When you have to wonder every other month if you will still be there a lot will be just quick and dirty ad hoc solutions.

It is still not easy to find good people, but the truth is: HR are not the best people to decide who is. If you have good people already and they don't try to build an empire, they will select good people. The danger there is only that they will select people that are too much like themselves or they will select the worse candidate because they fear about their own standing etc.

You should fire lightly early, and fire hard if late.

Where I live you have a month when starting were you can be fired with 2 weeks let-go period, after which it takes 3 months or more.

I think you should have an intensive monitoring/evaluation in the first month, and fire lightly if you think there is a mismatch. After you have determined an employee fits you, if they stop fitting you try to figure out why, help them to fit again (are they the kind who gets bored, needs new challenges?, do they have problems at home? If you found someone who fits I think it is more efficient to make them fit again then try to find someone else who fits). I believe this is the way to go even in areas where you do not have the rules about the dismissal period that you have here.

More accurately: such an environment requires paying a premium to compensate for the decreased job security. (As with any market, employees expect to be compensated for their higher risk with higher returns.)

Netflix, for example, and Amazon (to some extent) have gotten away with this model but the vast majority of companies are simply unwilling to pay top of market.

Additionally, in many markets there's no such thing as 'firing lightly'. Here in Scandinavia firing people requires a significant amount of work on the part of the manager and HR.

In Germany new employees usually have a probation period of 3 months, during which companies can pretty much fire them at will if they don't perform as expected. I think that's almost always enough time to assess whether someone is a good fit for long term employment.

That's what probationary periods are for.

Also, no one deals with technical debt if they don't stay long enough to suffer consequences of compromises, so the quality will decline over time.

> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire. If there were, everyone would be using it.

I can say with confidence that certain resume features are incredibly strong predictors of interview performance (at least in the related measures, in the field and/or team I'm working in). I'm not HR, but your statement seems to also deny the possibility of this empirical observation.

> Given this, why not hire lightly and fire lightly?

Because it would be a complete dick move to have someone relocate (possibly involving change of country) and then fire them 2 weeks later? That's not even looking at the formal, bureaucratic or training overhead, or any of the other factors in this industry that make hiring and firing a bit more complicated than handing someone (and later taking away) a hardhat and a shovel.

Can you share what those resume features tend to look like?

Disclaimer: This is for a quite particular set of positions, and it's not in the US (so I have no reference on US GPAs). The EU supply/demand situation is also different from the US, so your FAANG mileage may vary.

Here are a few (at a level of detail that I'm comfortable sharing):

- Outstanding (in either direction) performance in certain subjects in school is a very strong predictor.

- Cover letters are mostly noise, but incoherent, overbearing or careless ones don't go very far. Lack of curiosity and interest is also a negative predictor (but these are of course easily feigned).

- Relevant extracurricular engagement and/or industrial experience begets more well-rounded engineers.

Let me be clear though that we consider applications on more than these factors. These are just predictors, and sometimes they are wrong. I have interviewed actuaries that couldn't calculate dice roll probabilities, C++ programmers that were confused by the class keyword and C# programmers that hadn't heard of virtual.

Do you hire experienced people too? I someone has 35 years, does it makes sense to judge them on school results and extracurricular engagement?

These are people who were in workforce 10+ years.

You are correct to observe that what I listed is primarily relevant to people who have less experience. While prevalent, that's not exclusively the demographic of our applications, and of course the weight of these factors is reduced for someone who has been in the workforce for longer. But they remain good predictors nonetheless.

"C# programmers that hadn't heard of virtual"

No offense, but that is a very silly thing to consider "foundational" in 2020. The heavy, all encompassing use of inheritance patterns these days is much less prevalent than 10 years ago, even in the C# community.

Extremely silly, I got a C# job recently after I went about 7 years without touching C#. I forgot virtual existed until my first day when I had to look it up. I would have failed the "what's virtual?" question, yet I am very good at my job (just got a big raise).

I'll add a few more I've noticed over time.

Certifications are a strong negative predictor. Someone who turns up with a lot of corporate certifications is unlikely to do well when asked difficult questions.

People who have only ever worked in low grade or low tier banks tend to do poorly. There are a few exceptions. GS has good technologists. Most big investment banks have a few programmers (older ones usually) who have solid experience - but who are unlikely to put any creative effort in and may have a "here's why it can't be done" attitude. May not matter for some roles.

People who list a very large number of short roles tend to do poorly.

People who give a list of projects in their resume experience for which they only assert they were part of the group that delivered it, without concrete claims of tech leadership, often do poorly - generally it indicates a "hiding amongst the crowd" problem in which someone is a poor performer and attempts to hide lack of any actual achievement by reference to group projects.

People who list obscure programming languages as a skill do relatively well, people who list obscure products as a skill do relatively poorly.

> I can say with confidence that certain resume features are incredibly strong predictors of interview performance

But what about actual job performance?

See above.

Does interview performance have anything to do with job performance though?

In light of this particular team having been very successful at recruiting (i.e. extending offers based on interviews) and retaining people that succeed at the challenges they face for the better part of a decade: Yes.

But again, I'm not claiming this to be universal. Just that it's possible.

> it would be a complete dick move to have someone relocate (possibly involving change of country) and then fire them 2 weeks later

This. I hold a slightly lower bar (meaning more willing to take a chance) on someone who is unemployed than if they're working now. Worst case if I take someone unemployed, hire them "light", and they don't work out is they're back where they already were, whereas if someone quits their job to come work here and gets fired, they're worse off than they were before.

Shouldn't that be the employee's choice, not yours?

You're still interfering with that unemployed person's job hunt.

I’m not sure I understand. Someone is unemployed, applies, interviews just a whisker under our normal bar, and if I offer them a job, I’m the bad guy in this story?

> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire. If there were, everyone would be using it.

No, it's just the companies using it keep that data to themselves. Their hiring process is then a competitive advantage.

Very insightful. Indeed, I would.

Ah yes, a claim that can never be disproven, because any evidence produced would just show the company is not an example covered by the claim.

But I do know some companies who have a competitive advantage in the tech market, and their hiring is not great.

It has the same amount of evidence as all these "hiring is broken [for the most profitable and transformative industry in all of human history]" posts.

If hiring in tech is broken, then all of hiring is broken, whether it's fast food cashiers or aerospace engineers. Can you seriously claim that any of those have better hiring practices? From personal experience, I can say they are far worse.

That's not the ideal response if you actually wanted to defend your claim about the internal data and special competitive advantage rather than just abandoning the claim.

Seriously, I would say that the competitive advantage of a company comes primarily from its product and the vision and competence of the company leadership. You can't just hire your way to success without that foundation. Bad leadership can destroy the greatest employees.

The company founders almost always come together in a very informal manner, and often the early employees do too. And that's the basis of the company. It has nothing whatsoever to do with some special snowflake data-driven formal hiring process.

Indeed, I would argue that once a company becomes very large and successful, they become bloated and bad at pretty much everything, including hiring. At that point they have to get better by acquiring other companies, not by hiring through their standard bureaucratic processes. The competitive advantage of giant corporations is the ability to swallow other companies whole.

Those companies my must have ex employees by now. And no one has spilled the beans?

At least for startups, engineerng hiring can definitely be a competitive advantage. (4x startup founder here).

Google basically admitted that GPA and where you went to school don't matter and do not correlate with job performance. And yet here we are still optimizing on the Ivy League. It'd be a safe wager that algorithm whiteboard interviews don't either except that they screen out folks who don't pass the interview. So because everyone has that baseline, that baseline is by definition meaningless. The other side of the coin is that performance management is also just as bad.

Work results are only sensible measure for the work performance. Easy hire and fire is one solution. Also a short trial period at the start of the work agreement should show if results happen or not (fg. 4-6 months).

Yes, because those two things are related.

I don't know if it was ever made public, I think I heard this story via backchannels. But Google did a research project at some point to discover why GPA/interview performance in general did not seem to correlate with post-hiring job performance. The project was never allowed to reach full completion because the direction their data and investigation was going looked like this:

"Our hiring process does not correlate with post-hiring job performance because we have a large and measurable bias in interviews in favour of Ivy League candidates and women."

In other words, GPA didn't correlate with job performance for Google because it was a confounding variable: they were selecting the sample for top tier universities (which select on GPA) at hiring time, but not when it came to promo reviews (which had a different process and anyway, less rhetorical ideology involved than hiring, at least at that time).

It’s worth reading the source here:

https://www.nytimes.com/2013/06/20/business/in-head-hunting-...

> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire.

I don't have stats but anecdotally, hiring by personal reference seems by far the most effective way to hire.

> Given this, why not hire lightly and fire lightly?

Because it's a huge financial and organisational burden on the hirer and an even huger (sometimes existential if you live in the U.S. due to health insurance but that's a whole nother rant) burden on the hiree.

>Because it's a huge financial

It shouldn't be. From my experience, the financial cost that comes from increasing the number of people you fire after a few weeks is less than what companies spend furiously guarding against making a bad hire.

>and organisational burden on the hirer

There's no reason this needs to be the case.

To add to this, as a senior employee, bringing someone junior up to speed to the point where they can make actual contributions is a big consumer of time typically. Less so for employees who come in at medium rank but even then there is a hand holding period as the new person gets used to the company culture.

In a sufficiently large company you could have a specialized onboarding team, but you’d need to be careful that the specialized team doesn’t get isolated from the technical culture and needs of the rest of the company

If there was no meaningful data then a random hiring process would be optimal. This is obviously not the case.

It's not totally obvious to me that random hiring would be worse. Most people can do most jobs, given the right experience and opportunity.

Right. It logically follows that random applying would be a good strategy: selecting randomly from top-paying jobs, you would reply to job offers for e.g. software engineer, brain surgeon, architect or hedge fund manager, to increase the chances of being randomly hired.

That is effectively how I got my last 2 jobs in software engineering. Picking a few keywords and spamming out your resume seems the only way to match the equivalent efforts from recruiters.

>That is effectively how I got my last 2 jobs in software engineering

Are you saying that you had no qualifications at all before? Because that's what they're saying, just applying for any high-paying job, even if you don't have the requirements.

I mean my anecdata to serve as a realistic example of the hyperbolic concept of just wildly applying anywhere at random.

In both cases, I was switching industries (from ASIC design to IoT cloud software to data analytics). So, I arguably had some qualifications and arguably none at all.

Can you tell how fast did you acclimate and how did your employer look at this?

>It's not totally obvious to me that random hiring would be worse

Does this have some implied conditions I'm missing? Because to me it seems absolutely obvious that filtering for people who have experience/a degree will vastly improve your outcome over true random hiring.

A lot of things seem obvious but aren't actually true. Plenty of people with degree and experience coasted on nothing, or have personalities so abrasive as to make working with them all but impossible. Disappointingly I have poor experience with degrees actually instilling any sense of diligence, or even research capabilities, thus making them questionable even when recruiting for fields such as R&D, where it should be truly an obvious choice. At least a retrained nurse has some basic curiosity and a critical eye, though it's a bit of a shame, what with the nursing staff shortage.

And, also personally, I've had great experience with people who migrated from absolutely unrelated fields (e.g. several people who retrained - within same company - from support into software development). Similarly I had great experience with fresh interns turning out to be brilliant theory people.

I propose the Monte Carlo hiring process: heads means the applicant is hired without question, otherwise they get to go through the regular interview routine.

I like it. Exploration vs exploitation is used in a lot of optimization algorithms (and in reinforcement learning) meaning that there's a low probability that the agent just decides to make a random choice instead of making a greedy choice.

That would ensure also diversity of staff. Very good idea.

Why do you think that? Isn't it rather sexist/racist to argue this?

The candidate stream is itself not "diverse" (assuming you are using the normal meaning of the word of lots of women+racial minorities). Simply randomly making offers wouldn't alter the outcomes because those people aren't being de-selected by skills testing to begin with - unless you believe the 'diverse' candidates are less skilled programmers.

> This is obviously not the case.

Isn't it? While machine intervention might be able to weed out the most obvious low-quality candidates, I don't think there's any strong evidence that most hiring processes actually select the strongest hires - just that they, out of the set of candidates that progress to the point of human intervention (i.e. interview), select a decent one.

The problem is that interviews don't scale well, and some kind of automatic culling of the field of candidates is necessary. Engineers, managers, etc all want to feel that whatever machine solution (keyword searching in resumes, applying AI to recorded video statements, HackerRanks) they select is better than random - but there's no incentive for them to check that that's true.

Obviously randomness + interviews is better than pure randomness, but ultimately hiring processes are still pretty random. If you need to weed out most of your candidates, you can't do much better than throwing most of their applications away - and companies end up doing just that, only they cargo cult an "automated process" that claims to do better.

I would say that a random selection process would be worse than most hiring funnels that I've seen. This is doubly true considering the abject terrible quality of the lowest decile of candidate in software engineering. (The "spray and pray" resume machine-gunners would be way over-selected by any hiring process that relies on randomness when the rest of the industry is trying to select by interviewing.)

Has random hiring often been tried? I've never heard of anyone doing it. Not sure random hiring would be optimal, but it would be interesting if there were some data, doubtful it exists, though.

A couple places have tried open hiring (first come, first served) for some low-skilled positions, but with good initial results

https://www.fastcompany.com/90462746/the-body-shop-will-star...

> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire. If there were, everyone would be using it.

But every company in FAANGMO does use the same (or at least pretty similar) hiring process.

Yep. And there are zero large, successful tech companies hiring SWE's with anything other than Leetcode-style interviews.

Hiring is doing just fine.

This isn't true, depending on your definition of large. For example, Stripe is known to have an engineering interviewing process that diverges from the typical leetcode-style questions.

I think hiring is "doing just fine" and there's room for order-of-mag improvements on precision/recall.

Just because it's widely used doesn't mean it's good.

Source: they let me interview people at one of those fancy FAANGMO places (and I don't actually know what I'm doing). Interview success comes down to how lucky you get in the loop imo.

^ I completely agree. This article [1] fully convinced me. And it prompted me to write my own post [2].

[1] Some Thoughts on Interviewing and Why We Do It https://jwongworks.com/blog/2018/07/24/some-thoughts-on-inte...

[2] Screening developers should be easy https://acjay.com/2019/05/16/hiring-developers-is-easy/

> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire

This is a manifestly absurd claim. It is possible to improve the outcome of a hire, and almost everyone is using those kinds of processes. The firm I work for does an excellent job filtering people for skill and compatibility. The trick is that if you want to capture the top e.g. 10% of applicants, you're going to have to pay them several times more than the median applicant. Most companies are too cheap to attract distinguished talent.

>The real truth is: hiring is a crapshoot. For any position and any field. Software engineering is no different

I tried building spaceship and failed. Fiend of mine tried building a spaceship and failed too. However, I know of people who built a spaceship. So is building a spaceship a crapshot?

Reality is: 95% of HRs are incompetent. That's where randomness comes from. Take a random man from street and assign him for HR role after a brief training -- you'll achieve a similar performance to those 95% HRs.

Everybody's talking about hiring the engineer, but so fewer people talk about finding a good HR that would help you to solve a "simple" problem: how to find a good engineer with small experience and small salary that would not leave your company for the next few years and would become after several months just as good as 70-100$/hour engineer hired from top software company, while later most likely still gonna need some time to adapt to your product to become efficient.

Because hiring has costs, this requires substantial risk tolerance, and is more suitable to a bigger corporation that can bear them. It will also give you a reputation — though some companies seem to manage quite well even with an iffy reputation in this area (e.g. Netflix). You need a certain amount of prestige to pull that off, I think.

Being hired from my experience hasn't been a crabshoot. Everywhere I've been hired it's been a success.

Hire people with multiple pages on a resume with a variety of experences is a good strategy increasing your odds.

id also say it works out okay. its just long and tedious and it feels more painful than it should - but really its not all that bad in general.

I see very toxic people being rejected that think its because of their gender/color/whatever <= lots of complains but the process work

I get rejected sometimes because of misalignement from me, or from them <= feels bad man.. but the process works

I've never been hired to a place where I found that this was all a terrible idea, in 25 years. Theres been places better than others, theres been adjustments after hiring.

I've seen places hire people using simplified processes and the candidates were unsuccessful and unhappy after being hired <= process fail

I think a lot of us suspect that just choosing applicants randomly would probably have a very similar outcome to what we do now, but nobody wants to be the first to try it.

The above statement is really just validation that the current process sucks so much for everyone that it's conceivable to replace it with random choice. However, it's not actually true (for reasons explained in the blog) - as a hiring manager, I am confident you couldn't ship "Hello world" by hiring randomly.

> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire. If there were, everyone would be using it.

Word of mouth recommendations, and yes everyone uses it.

> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire.

Daniel Kahneman analyzed a bunch of data that lead him to concluded that the typical interview process did nothing to help select the best candidate. There's a chapter about it in Thinking Fast And Slow [1] and the advice he gives is summarized in this article [2]. I remember thinking after reading this book that it was just a matter of time until everyone everywhere would be denouncing interviews but here we are - old habits die hard.

[1] https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp...

[2] https://www.businessinsider.com/daniel-kahneman-on-hiring-de...

What are you talking about? The article is nearly the exact opposite of what you claim. It says:

> A vast amount of research offers a promise: you are much more likely to find the best candidate if you use this procedure than if you do what people normally do in such situations, which is to go into the interview unprepared and to make choices by an overall intuitive judgment such as "I looked into his eyes and liked what I saw."

the typical interview is a series of unstructured intuitive judgement calls - Kahneman suggests something more like a survey with a strong emphasis on its calibration and isolating/controlling biases. Thinking Fast And Slow goes more in depth on how terrible typical interviews are - the article merely summarizes the advice he provided for what to do instead.

Reply on news.ycombinator.com