How to Hire Great Engineers
autoiterative.com
autoiterative.com
- They are probably not going to be able to hire "great" engineers
- Even if they get a couple of great engineers, they are not going to be able to create an engineering culture where those engineers can do great work.
Instead of looking for "greatness", most companies in this position should simply settle for competence. Hire people who use boring technology and build systems that do things simply and directly.
What I do know is that company processes burns out all sorts of people - experienced or not. From stack ranking, agile micromanagement, unrealistic deadlines, office politics, bad bosses... These things are prominent in nearly every company, especially in top companies like Google.
I agree with the parent post, even if you by chance somehow hired fantastic 10x developers to your team, the environment in which those developers will be working in will probably burn them out quicker than just working on 'boring' software.
Best people do thing that matter.
There are fewer things that matter in big companies.
It's not negative. It's pragmatic. If you even happen to just encounter one of those people (they're rare by definition) then do what you can to hire them, but the truth is that the vast majority of companies producing software wouldn't even know what to do with them if they could attract them.
Stable, predictable, performance has a ton of value too - and you get a lot of that from 'regular' developers.
A generally great engineer to a startup is somebody who can ship fast, make good enough decisions and trade offs, can think on their feet, and is a jack of all trades.
A generally great engineer to a big company is somebody who can work well with others, can follow process, can work within huge code bases and abstractions, and follow architectural best practices.
These two do not necessarily overlap.
Best engineers are working at high growth organizations. Pay and position do reflect competence if you are looking at that demographic.
Google has brand recognition, and an enormous recruiting pipeline so they can hire one in 50 or 100 candidates.
They are better able to select for great candidates because they have more data and much more sophisticated system for identifying talent.
They pay very high salaries so they can retain top talent.
The average candidate that applies to Google is going to be a better than the average candidate that applies to a small/medium shop.
A small medium shop that tried this would be chronically and severely understaffed with a high turn over and would probably still fail at finding the best candidates.
If you are a small or medium sized shop and really need top level talent the best thing to do is probably just hire googlers and netflixers and pay them a market rate of 400-500k a year.
But for most products it's not worth it because the end product wouldn't be materially different enough from an average developer building it to justify the cost.
Also most small to medium shops could improve engineering quality by taking the Netflix approach which is getting better at firing people, and compensate them for the lack of job security with cash.
We need to make money but it’s my responsibility to ensure we do it safely. Incredibly difficult with adderall addicted developers who shine bright in politics.
Many of the higher-ranking engineers I've seen at big companies weren't necessarily brilliant, but they were all well regarded.
This is great engineering though. Isn't the trick to find engineers who don't build nests for themselves and create strategic technology problems for your company, especially if your core business ins't tech.
In software world, you can also satisfy 99.9% of cases with existing approaches. You mostly don't need great engineers for that.
Unlike other engineering disciplines, software has a constant pressure for pseudo novelty and fad adoption. One dimension of "greatness" is resistance to those temptations and pragmatism in the face of constant change. In that regard, lower quality engineering talent often correlates with increased tech churn and defects, even given a simple business domain.
Tl;Dr engineering excellence doesn't only imply innovativeness, it also implies discipline.
Why not roll your own? I am constantly amazed by companies moaning "We have a skills shortage! We have to get people from overseas."
Sure, get people from overseas if that works. But bloody train people to do what you need! For the FAANG, train hundreds of them!
Sure, you may want penalties for people leaving early after training or bonus for hanging around after training (like you get bonus stock 1-3 years after the training) but HR people, get creative and do your damn job! Your job isn't just hire&fire.
Great engineers use boring technology and build simple and direct systems.
Boring technologies seems like a good idea until you realize the market place for software people rewards people with cutting edge experience.
> The grading is automatic. I think is ok as _one_ metric, but I wouldn't want it to the be the _only_ metric that is being looked at. ICs aren't really ICs. You can be a JIRA champion, knocking tickets into the done column at the speed of light, but Great Engineers do more than just that.
Great Engineers build up their team, and help others grow. They know how to push back on unrealistic asks. They can incite help when they get stuck. They can translate complex issues into digestible bites. Writing good code is the bare minimum quality of being a Great Engineer.
The main barrier I see to hiring Great Engineers is that you really need to hire Great People. But a lot of hiring practices are taking all the human aspects out of hiring (except for the "behavioral" questions, but even these can be gamed like system design or leetcode). Great People worthy of hiring are driven, passionate (not "spent all my waking time coding", but "truly believe in the mission/goals of the company/project"), empathetic, intelligent, and trust worthy. None of those traits are easily discernible from random logic puzzles.
I don't know what the solution is. At some point there has to be a "can actually write code" litmus test, but I don't believe any of the current methods work well to prove that. I think take home tests are closer to the "chopping onions" idea, but only require commitment from the applicant, not the employer, which leads to a lot of wasted time.
Personally, I think temp-to-hire or work-for-a-day type models could work, if there was some standards to prevent abuse. I would rather sacrifice a few days to a week to see if I actually like working somewhere, and to prove to them that I can do the job, then look at CTCI ever again.
Interesting, why do you think was this the case?
There’s also the aspect of working with the rest of your team on whatever problems you are solving and not siloing yourself to just the problems you’re working on.
That and when you’re hit with new and interesting problems that have never been solved before, you cannot look those up in the leetcode manual and memorize them to solve. You have to actually create something on your own and apply whatever you’ve learned from programming in the past.
It’s like playing a survival video game with the detailed how-to always at your fingertips versus being in the wilderness having to survive on your own wits without a guide.
Any sensible developer learns early on that mostly the coding is the easy stuff - what Is the problem I am solving here?
- using http apis and transforming the resulting data
- using database queries and transforming the resulting data
- navigating a large codebase
- figuring out what tasks are important to accomplish
- researching things you don't know
- debugging a complex issue
- evaluating tradeoffs
- general engineering intuitionthanks, can be useful tips for my future staff recruitment.
Well, first make sure there actually is some "mission" there to "believe in". In 99.9% of the cases people work on perhaps technically exciting but impactwise boring industry projects, whose main mission is to make money. Requiring passion is a euphemism for being over-workable. That you'll work outside working hours, you can be called during vacation etc, because of course this job is the mission of your life, right? Otherwise, you're not passionate...
In practice, these "passionate" people are often young and naive or just really ambitious and work their ass off to build their resume and will be on their way soon enough.
A good engineer contributes high value and knows this. And therefore will make the company pay up, and will not take any bullshit. And being that good, they will be picking where to work and you as the company compete to get them. In these cases the interview is about the engineer judging the company and looking for exactly things like the requirement to "believe in the mission" or whether the conversation is on a professional, adult level.
Passion, imo, is an attitude - and is reflected in quality of work and teamwork. I do think it is often used to trick people into getting less than they deserve, but doesn't have to be the case.
When I say I want to work with passionate people, I don't mean people who are willing to work 80 hours a week for 10 dollars an hour because they love coding or want to solve world hunger. I mean I want to work with people who give their best, put the user/experience first, and are genuinely motivated to do good work. This doesn't mean you have to fake smile, or get taken advantage of, it just means you have bring some positivity to the table.
This is the issue with signalling passion - it causes people to drive their expectations of you higher. You have a higher chance of not living up to someone's standard.
It's kind of mutinous, but it's not like there's anything to be gained from loyalty to management who aren't loyal to employees.
It goes both ways. The current equilibrium, at least in the US, is that (SW/IT) engineers leave after ~2 years, and it's causing quite a bit of friction. But it's hard to move out of the equilibrium.
I don't think it's so much an equilibrium so much as a battle, denoted by "friction," but I'm sure it would come down to each saying the other one defected first. I think we can figure out where the disloyalty started, calculated from the first principles of capitalism.
That's definitely true in the general case. Having passion means wanting to do the best overall thing because you care about the thing going well. The best overall thing is working hard to prevent long hours which lead to burnouts. If you have managers and senior leaders who share the engineers passion, this problem is self-correcting.
Source: I frequently lecture our (junior esp) new hires about working too many hours. Especially during incidents. Yes this is exciting. Now go sleep and see your family. Come back when you are rested and ready to play at full speed.
I also have uncomfortable talks with people who frequently say "meh, close enough" about whether they actually want to be here. My team performs at a consistently high level.
Yes. Makes me think of Zappos, the shoe store. They're big on demanding employee enthusiasm.[1] But in the end, they're just a shoe store.
I've seen the contract-to-hire stuff come back up recently. There are a couple big flaws with that model, beyond possible abuse, though.
1) You basically are excluding yourself from any candidate that still has a full time job. It would be crazy for them to leave their current gig for a place that might not actually bring them on full time.
2) It doesn't scale, bringing on new team members properly takes time and effort. Not only are you potentially wasting their time, you are also wasting your team's time, and you can only do this kind of evaluation so many times.
I do agree though that the standard interview model doesn't make sense. Memorizing leetcode problems doesn't map to good engineering, and take home problems filter out a lot of good engineers that won't bother spending the extra time.
I fully agree with you about temp-to-hire or work-for-a-day models. And I think if the company has such a process in place, it is the sign of a well-developed culture. It is exactly the point of the blogpost -- evaluate using the right metric for the job. However, this is rare in practice, and I would say for the same reason the well-established culture is rare: it requires a lot of work to be done by the very same great people in advance. It is a chicken and egg problem. I saw countless times in multiple companies the signs of immaturity that prevent them from giving people access to their production: secrets in git, shared accounts, lack of established onboarding and offboarding processes, no processes for credential rotation, absence of audit logs, legal issues, you name it. The majority of companies can not afford a work-for-a-day option.
Maintaining the dedicated challenge is another option for them, but it has its downsides, too. In our experience, the main one is that you either invest in creating the grading infrastructure upfront or spend a lot of time manually grading the submissions. Checking the correctness of the code is a hard task unless one can actually run it and see how it performs under a barrage of tests. I'm not saying there should not be a code review, of course. I am saying that the automated assessment of the code should be done before you start spending human time. Machines are cheap, your engineers are not. And at this point, one doesn't need AutoIterative, but has effectively reinvented it :)
We walked this path ourselves over the years. We did fizzbuzz style interviews, that was suboptimal. We graded challenges manually for a long time, which was tedious and error-prone. We made all the mistakes and tried to fix them. We were using the wrong metric.
Qualitative improvement was when we started using the right metric, and automating its assessment was the next logical step.
There is your problem. You are testing for something unrelated to the job.
That's understandable any one with that much proficiency on leetcode would be wasting their whole day on that site than building real things.
Everyone criticizes this, including me before I decided to capitulate to the system and do it myself. But it's actually kind of fair? Everyone knows the game. Some people will choose to train hard enough to play, some people will not. That is a signal in of itself as to whether the candidate will work well in the big tech company environment.
But just as this post says, as a hiring manager, you'll get very little insight into how good the candidate is until after you've hired them. This does serve as a weak signal for dedication, but:
1. You don't want to make your hiring decision only based on dedication.
2. There are other, better signals for dedication.
The fact that one depends on a proxy measurement which has little correlation with actual results is the core of the problem here.
My compensation increased by at least $150,000 a year.
Best ROI I've gotten on anything I've done in my life.
I tell that stories to people considering doing it all the time, you'd be amazed how many people hear my experience and say, yeah, naw I'm not going to spend that time.
¯\_(ツ)_/¯
What was your question bank.
At the end of the day, it was the ~40-60 hours I put into Cracking the Code Interview that got me the FAANG job, the side project was barely a blip.
That's a weeks wage on what is essentially a lottery ticket, albeit with good odds. (Having done some freelance work for a friend in my Spare time, finding 40 hours free is going to take me weeks).
It really is a lottery ticket. Luck plays a ridiculously large part of the interview.
Half an hour later wandering done the street a very elegant solution came to me. I wouldn't say I am bad algorithmic stuff, but as I never practice it's pretty random whether I will get during an interview. "Luck" as you say.
In a smaller company this is not true though. If you only have 10 engineers there is a good chance that none of them is qualified enough to solve the current problem so even if they all are extremely loyal and cooperative - the problem will remain unsolved. Even with a 100 you might end with just ~10 experts who are going to be overextended, burn out and quit. So, while loyalty and teamwork are still important, you really, really want to test for the actual expertise in a small-medium company. Unless you don't foresee any hard problems, then hiring for loyalty is the best you can do.
Except that a company that's 10 times as big is working on 10 times as many things. The idea that a huge company can afford to hire just anyone because they're so huge and they only need a few smart people is ludicrous. If that were the case, these companies wouldn't feel the need to pay so much.
The scenario I have in mind is something like this: you make an app, your team manages UI, backend, whatever, just fine - it's simple and is 99% of the job. You release your app and a million people download it, next day you are flooded with crash reports, it crashes for 25% of the users!
Megacorp: calls a special team of greybeards who spend 24 hours digging and find it's a bug in a driver for some popular chipset. They write a fix in another hour, file a bug with the popular chipset manufacturer and return back to playing WoW.
Small business: panic, panic some more, post questions on stackoverflow, really panic, find a bogus solution, release a patch in a week which makes the app crash even more because it's rushed and is actually breaking things by design, go back to panicking. The app gets bad rep, even people who had no problems are dropping it, its reputation is destroyed. The small business tries to pivot and shuts down in 6 months.
Problems like this do not happen often but when they happen it may end the business so you don't need a lot of resources to solve them but if you have 0 then you are not going to last.
And you don't get better engineers by hiring just anyone who comes along.
And then it looks like this: when hiring someone, the company measures them by one dimension and then expects them to perform on another one. I did get some backlash for comparing engineers to cooks, heh, but the main point is and always was about choosing the right metric.
Many engineers are also looking for jobs while they currently have one, making a half/whole day "trial" very difficult or impossible. Asking people to commit their weekend or free time is also discriminating against those with families, social lives, and those who just like leaving programming behind when office hours are over.
The article only talks about algorithmic interviews. There are plenty of other interview styles that delve into more pragmatic problem solving that still aren't perfect, but work much better.
They can ask the current team questions via chat. Get some real code into PRs and prove that they have skills.
Personally I don't even apply to jobs anymore. I am tired of 6+ hour code reviews that I don't get paid for, and no explanation when I get passed up.
I would much rather be a contract worker and get paid at least something while doing actual work, over building a search engine for star wars characters.
I'm exactly like what you describe. When I leave the office, I want to camp, rock climb, and prep for marathons. I don't want to spend more time coding. That's exactly what makes me absolutely disgusted with applying for new opportunities. Eight interviews with various people, just to get ghosted. Time I could have been improving at other passions. Instead I'm left with no time, no indication of what I can improve on, and no job.
Genuinely curious here, I only know one model and it works well enough for me, but you seem to have a different one and I'd like to know more about it.
Obviously, should my current security evaporate, I would resolve to deal with the process.
I'm not entirely sure what you're confused about. Nor, what model you refer to.
Thanks for mentioning this. Once you have a family at home it gets unrealistic to not work while looking for work. Especially in countries where your contract can usually require months of notice and preclude unemployment in case you quit.
It's ironic that the article points out the error in blindly imitating others and then we get this entirely apples and oranges 'recipe for effective interviews'.
What are the fixed costs of having a 'live' chop the onions and shoot the breeze on slack for software companies? For a restaurant, they simply need a bit of tabletop for the 'new workstation'.
- All onions are pretty much the same in one industry; in our industry, we famously have 'a new way to slice onions' every other day! Let's not even discuss the variety of knives ..
- What 'secret sauce' info is disclosed during the 'live' getting to know each other? Can we just put candidates in company slack and check out how it works? Seems problematic.
> They would put you to the test for like 2-3 days to see how you perform
And pay you. And give you some level of feedback along the way, not just "here make some salads" and then "nah nevermind" after 3 days of hard work. And you get to work with them and decide if you like working there too. It's not a one-way street like these bullshit homework interview pipelines.
Unlike in food, you're not going to be productive with real code for a few months. Unlike in food, just doing the coding is only 40-60% of the job. Unlike in food, it's hard to improvise code in a language, framework, test-style, knowledge-domain, or other toolset you may not be familiar with. Hiring for restuarants is not a relevant comparison.
This just seems like the typical unpaid homework assignment--which is usually a couple softball algorithm questions mixed with some arbitrary glue-code riddles--with some CI-flavored slop on top.
I've hired several engineers who get productive within the first week. Most manage to get in a PR and deploy to production by the end of the first day.
If it's taking a few months before the candidate can get productive, something has gone horrifically wrong.
Agree 100%. And in most cases it probably isn't a problem with the candidate. It's a problem with the existing software and development process. It shouldn't take someone months to become useful on a new project.
It's also a good sniff test if you're interviewing. Slow and steady might be up your alley too. There's definitely a hierarchy of security and compliance needs depending on the org, but many orgs loose the forest for the trees.
There's a big difference between "deployed their first PR" and "getting productive." The latter implies the engineer knows or has figured out how to use your source control, CI, and deployment processes. That probably is something most people can manage in a day, with proper instruction, provided the code change itself is simple enough. The latter means they're familiar with large parts of the codebase and are able to make changes, and have an idea how the major parts interact. This is what takes a few months.
As an interviewer, I think they are the best at assessing the coding aspects of job performance. Not perfect, but better than high-pressure in-person whiteboard/coding tests.
As an interviewee, I like the flexibility to work on it as I have time (rather than using up PTO).
The point of using algorithms isn't to test your algorithm skills (though using job relevant algorithms is a good idea if you can) but to test your ability to translate requirements into code by eliminating the quality of requirement specification as a variable. That is, because algorithms have an unambiguous specification, failing the coding assignment due to poorly specified requirements should impossible (or much less likely).
For the non-coding part of the job, I find a (relatively short) structured interview works best.
My real problem with take home assignments is that while they take away the stress of being put on the spot in front of a whiteboard, often, the reward for doing well on them is... being put on the spot in front of a whiteboard. If anything, it might get one past the phone screen, but phone screens are not the biggest problem in SWE hiring.
Ha! That can be true. So for me personally, I had candidates do a little bit of whiteboarding, but all I do is ask them to walk me through the execution of the algorithm they just implemented using minimal input parameters. Really what I'm looking for is the ability to explain how their code works (and to make sure they didn't copy someone else's work).
Ask the candidate to talk about, solve, and explain their favorite whiteboard-style problem. Ask them for something they were initially intimidated by. Maybe ask them a slight twist of it. We all know candidates study riddle problems, so let's both take advantage of that rather than giving an anxiety-test even if it's phrased as low-key.
Right. This practice is not better than an onsite. It's actually worse. Not only do you have to put in arbitrary amounts of time, you're not able to do any actual assessment of a prospective employer. You have nothing to go on other than a random problem statement which the company could have perfected over hundreds of hours and interviews. It's giving the employer all the leverage while also doing work for them for free.
To me an on-site is more serious and shows commitment. You are asking the candidate to dedicate an entire day but asking the same to your engineering team. From my experience, engineers with multiple competing offers will go to an on-sites with certain companies that they would have ignored had they sent an online homework.
If some people find this process too burdensome, that's their right. But plenty of skilled engineers that I've worked with don't see it as being jerked around. At the very least, I find it less burdensome that turning coding into some sort of live-performance (which an in-person coding test inevitably is).
Personally, I don't judge an hiring process primarily by how convenient it is for me. I judge it by how likely it is to be a good filter. If the company I'm applying to has a hiring process that is effective at selecting good candidates (along a number of different dimensions) then I'm more likely to enjoy working with the people there.
From this perspective, homework assignments are a good filter, and so companies that use that approach seem like better places to apply.
You've taken all of the leverage and one-sided information, given nothing to the candidate, and you expect them to be okay with this without payment. Good candidates, myself included, will not jump through arbitrary hoops or suffer such foolery.
I have ample github history, 15 years of senior-level experience with good companies, and the ability to discuss solutions at every level in a tech-stack. I will not coderpad on my own or do unpaid homework. I will pair-program for an hour with an agreeable potential-colleague, but not with somebody who's hiring for other teams.
When you say jump, I say jump with me or I'll see ya later.
This is a huge part of it to me. In a whiteboard/pair-programming interview, you get the ability to judge the company and potential colleagues. In a take home, you get absolutely nothing out of it.
I have been on both the interviewee and interviewer side of this interview style, and I speak from both perspectives when I say that like this style.
> You don't have to actually confront a human being in front of you, understand the thought-process they went through in doing the work, answer questions about yourself, projects, or the company. You can say no by typing a bit.
Are you assuming that the homework is the only part of the interview process? That's absurd. Once they complete the assignment (which should be an objective thing to assess with practically unambiguous criteria) they are brought in to an interview. If they did a good job on the homework, they'll likely get the job. I would rather not ask them to take time out of their workday for their current employer until I think there's a good chance we will extend them an offer.
The interview is exactly what you describe: a chance for them to evaluate me, for us to have a discussion about their thought process regarding their code, about their prior experience, etc.
> You've taken all of the leverage and one-sided information, given nothing to the candidate
Not sure what you are mean exactly here.
> and you expect them to be okay with this without payment.
I also expect them to send me a resume without payment, and people can easily take as much time editing and formatting their resume as they do on the coding assignment.
I'm not categorically opposed to the idea of paying candidates who submit a (correct) solution to the homework assignment. That seems like a nice gesture. It just wasn't an idea that ever came up in discussion, and in retrospect doesn't really seem like it was necessary.
Honest question: If you were to get paid for a completed homework assignment, would all of your objections go away?
> Good candidates, myself included, will not jump through arbitrary hoops or suffer such foolery.
We had some really good candidates (and hires) using this process, but nice No True Scotsman.
> I have ample github history, 15 years of senior-level experience with good companies, and the ability to discuss solutions at every level in a tech-stack.
Good for you.
For every candidate, I would always assess the homework assignment before looking at their resume, or even seeing their name. I wanted to have it be as much of a "blind audition" as possible. We brought in people with weak homework assignments and strong resumes, and people with strong homework assignments and weak resumes (and other combinations). While not a huge sample size, the experience of myself and others on my team was that a strong homework assignment was a better predictor of a good interview and job performance than a strong resume.
> I will not coderpad on my own or do unpaid homework. I will pair-program for an hour with an agreeable potential-colleague, but not with somebody who's hiring for other teams.
If that's your preference, that's fine. As professionals in the same industry who both in good faith want to improve the state of interviewing, we can discuss the trade-offs between homework and pair-programming.
But let's not pretend there are no trade-offs. Time-boxed live programming interviews favor people who are good at working under pressure and who are already familiar the the technical stack. They filter out people who may be great engineers but are more deliberative or come from a different tech stack.
Let's say you hire 1 our of 10 candidates. Can you afford to give these types of assignments and manage them for 10 people to hire 1? Sounds completely implausible to me.
Can they afford to spend a few days coding or interviewing with each company? That's more than a solid month of work for the candidate.
But anyways, I'm not a hiring manager, so someone go out and try this and tell me why it's a terrible idea.
This kind of model is way too biased towards employers and gives them too much power to exploit, in an environment where employers anyways have a lot of power as compared to one employee (prospective candidate in fact, not even employee). This reasoning works well for companies whose job is to recruit or help recruit engineers. After all, they are being paid by the employer and not the candidate, so they want to make the process optimal for the employers. But as much as I like the doing the right thing for customer attitude, this seems like taking that to an extreme and exploiting everything that isn’t paying.
And the issue with that is that great talent will simply lose interest. Unless of course the company is willing to be extremely aggressive salary-wise or is a leader in a very visible field (SpaceX). Because they typically have have multiple offers on the table.
Especially if I'm in full-on hunt-mode, I might be doing a couple interviews a week. This is totally infeasible.
I'm all for ditching the canned CTCI rigamarole, but please point to something that respects the candidate.
The article introduces a problem and a solution that they came up with. Are you saying it's trash just because they have a chance of making money off it? It's ok to have a conflict of interest, imo - the work should stand for itself, and in this case I feel it was a good article.
By the time I got to the end, I realized it was an advertisement, but still felt I had learned something useful.
I would challenge the assumption that this is purely advertisement, though. In my head, this is not "buy our shit™" type of post -- in fact, there's no mention of buying the subscription anywhere. All there is is a link to a demo, with a valid challenge (not a fizzbuzz/fibonacci, but one you can actually have a lot of fun with), which we provide for free to anyone, w/o even requiring valid email -- it is up to you if you don't wanna reset the password later. Instead, the post is more of a "use the right metric -- here's our experience and what has worked for us" type of post.
Would you agree that this is a more fair assessment?
We tend to put more trust in end product in that regard more than any specific process (like quizzing them on the process to create a specific animation in After Effects or on interaction cost or cognitive ergonomics).
I assume it worked out, because he's still there.
I was a hiring manager for a long time. I seemed to make good decisions, most of the time (a couple of notable exceptions). I kept fairly senior-level employees on board for decades.
I don't know if I ever developed a "system." It was always important for me to establish as much of a relationship with the prospective employees as possible, up front; which meant treating them with respect, and being open and emotionally available to them. I found that an attitude of trust and respect goes a long way to figuring out how people will deal with you and your team.
Tests were worthless. I loved portfolios, and discussing the details of projects they had done. I did a great deal of homework before each interview; carefully reviewing their resume, and finding out about as much as I could.
From all that, I have learned to be an "open book," myself. There's really not much about me that can't be figured out with some quick googling. That's on purpose.
But this seems to be more or less an advertisement for their company. It may be a good idea and a good company, but I can't really say.
The candidate only has access to the initial test that gives them early feedback on how they are doing. The rest of the stages are visible only to the hiring manager."
I almost love this, except for the second paragraph. It sounds as if you can't see how well your code performed in latency, load tests, or edge cases. That does not seem like a realistic simulation of work, because normally one would need to iterate a little bit to optimize those things.
I mean I know I would prefer to just do an average job passing the core tests before I try to completely optimize it. Are the candidates even informed that the latency and load tests exist?
I think it could be okay if you just mention that is part of it and there is a straightforward way for them to do their own latency and load testing.. so hopefully that's the case?
The kind of "chef" you interview by asking to chop an onion or cook for a few days is not the type of chef you are probably going to ask to design the menu. You're not making the menu, it's an almost pure execution role.
One of the things that can really set a great engineer apart is the ability to design elegant systems and manage complexity. You're likely not going to find that out coding with them for a few days unless you make sure to index for it.
Ironically, bigger companies where much of the design is centralized would benefit from this approach much more. They can get a lot more out of a "grunt" engineer who does journeyman work and is careful about testing.
I think that this is great idea that you're trying this. I hope you stick with it and succeed. Because it would be great to be able to hire and be hired, where you're not competing on how cheap you are or some other demeaning metric.
Godspeed.
Just my opinion, but I think this line comes across as a little too condescending or perhaps paternalistic. I work at a small (40-50 ppl) unknown to you company and all of us (so far) are senior in our fields and largely have FAMGAN experience on our CVs. We’re not just cargo culting, but rather taking things from the FAMGAN hiring processes that make sense and using them.
I would argue that 40-50 senior people in one place would be rather an exception from the norm, and your company would be quite an outlier on the bell curve. If all of you are not just seniors, but also had first-hand experience participating (or even building) in hiring loops of the large companies, then you would have all the necessary knowledge of why those rituals were created, and why. The point of the blog post was that this is usually not the case, so the imitation is the only option.
Could you elaborate on your hiring loop? Which of the stages is the most effective and brings you the most value?
Cultural alignment No less important is the cultural alignment, the “Can I work with you?” question.
This screams all sorts of “isms” to me.
"The same happens with time-constrained problem-solving. That’s not how real life happens. People need time to think and reflect on the problems they face to build the right solution. Because of this, we do not artificially limit time for solving a challenge."
sounds very critical in that regard. If you have lots of time, that makes it much easier to get outside advice.
I once worked with a guy that was very difficult. People warned me ahead of time. I made sure to accommodate him and we got it done. Otherwise, he would not fit anywhere, he would be jobless and then what?
If you don’t respect the people’s right to work, you exclude them from society. And then what do you think it will happen?
On another note, if your code base uses very niche software and there aren't too many people out there who know it, what's the process for hiring an engineer that will be able to learn it? What do you look for?
So pay them, even if it's just a token amount. At least shows some appreciation for the candidate sacrificing their time.
(Oh, autoiterative people? Where's your privacy policy? Where's your opt-out of tracking link? For that matter, where's your company? And why is your domain registered via WhoisGuard out of Panama?)
Thanks for the questions, let me elaborate:
- I didn't have time to prepare privacy policy, but we'll indeed make one and post it. You're right, we should have it.
- There is no tracking. Never will be. We'll blog about why we're for 100% anonymity of the candidates in more detail, but for now: We don't collect names. We don't require valid emails to enroll in the demo. We don't use tracking cookies. We don't send HTML emails with tracking pixels, and we don't use any 3rd party service. We use cookies to authenticate you, and that's it. Javascript, social media buttons, or trackers or analytics of any sort on our pages are considered a bug. What we don't have, is a proper explicit privacy policy explaining all that.
- We're a European company.
- That's how Namecheap sets it up -- curious, why do you think it is a bad thing?I'll play devil's advocate and defend whiteboard interviews as compared to take home or online assessments. Take homes are not very respectful of the candidate's time and, especially if they are not time bound, often leads to a candidate wasting a significant amount of time on these assignments. Plus, there's always the possibility that desperate candidates would cheat and have the assignment done by someone else.
I would only use take homes and HackerRank style quiz for applicants that are from institutions I don't fully trust with quality or applications that are very spammy in nature, i.e from a college/bootcamp that has an abysmal application-to-hire ratio [1].
For candidates from serious institutions I favor a two-step approach: a screening interview with a light coding question and a full whiteboard interview. The screening's purpose is really only to answer questions about the hiring process and check that the candidate can write very simple code by himself when given a trivial problem [2]. Something not more complicated than FizzBuzz really.
A lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course, it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can't make any assumptions on what the candidates know.
People, especially non-technical, over-emphasis on knowledge of specific languages and framework and disregard computer science fundamentals, and I've been doing the exact opposite. I'm not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast.
Now, large tech companies optimize hiring at the college level because that's when everyone is on the market at the same time [3]. It's also why they have internship programs, to make sure that the best talent is already in the hiring pipelines a few years before even getting their diplomas. After that initial post-graduation job search phase, you'll see great engineers often spending several years at the same company and not even looking for a job. They are often the strongest candidates and totally invisible to recruiters.
Lastly, you can either really hope hard to find a diamond in the rough at whichever market rate you set or decide to compete with the companies that are actually attracting top talent. There's really no secret formula it's: compensation, interesting projects, technical career path and technical management. You can try to play games with CoL or market rate, but just keep in mind that if you do some of the top applicants won't even bother sending a resume in the first place.
[0] https://news.ycombinator.com/item?id=23989466
[1] https://economictimes.indiatimes.com/tech/ites/95-engineers-...
[2] https://blog.codinghorror.com/why-cant-programmers-program/
A few days of a truly great engineers time costs like 4K-5k. At least.
* hoard knowledge
* refuse to elevate those around you
* are frequently combative
* lie about your knowledge/experience
* seek to be right instead of discover optimal solutions
... then all of that diversity is lost or wasted.
It's not about diversity for diversity's sake, it's about sharing in diversity and using all of the differing perspectives to reach optimal solutions that your organization would be blind to otherwise.
I will try to address the questions tomorrow, I already see some of them require a ton of thinking indeed.
At this point, we decided that this is too good to hold only for ourselves. Thus, AutoIterative was born, where we also significantly improved the parts that weren't polished well enough and added features we lacked.
Some context: we're not claiming we have a silver bullet or fit-it-all solution as some people got from the post (and that's my fault because I'm responsible for that text, and blogging is hard, haha). And we're also not claiming we can replace _the whole_ of your hiring loop (that would be the silver bullet).
We claim that choosing the right metric can drastically improve the hiring loop, and we've built the tool that automates the assessment of this exact metric.