How to hire engineering talent without the BS
jes.al
jes.al
At least the reason why we have Leetcode questions is because Google did the research and came to the conclusion that those are were good at algorithms ended up being more successful at Google, and THAT'S why we are all suffering through LC. But now that LC has been gamed, I would love to see what the results are as to what makes a successful interview.
Google is beginning to show its age, I’ve seen a number of past coworkers get hired there lately that weren’t effective at their job, but I’m sure they crushed some Leetcode.
Yup. Goodhart's law strikes again. I am curious as to what the next as-of-yet-ungamed effective hiring metric is.
See also: college admissions. (No, that kid probably didn't do that volunteering activity because they wanted to help people, but because they were told it was a checklist item, as they were coached through college admissions. And they probably didn't learn anything, and they were coached on what to say about that in the essay. And, normally, actually productive volunteering probably wouldn't have looked like each one of those kids starting their own duplicate initiative so each could spin it as demonstrating leadership. And it's more an option for kids from families well-off enough that the kids didn't have to work jobs that would contribute better to their household income or college expenses. And don't get me started on how the well-to-do decided that "travel" would be a plus on applications.)
Then they give you your real chair.
It is probably realistic to expect someone to write something useful (although possibly small) in three days. It is less realistic to expect someone to write a useful component integrated in a large system that they have to learn, in three days.
Is that actually true? It's probably uncommon in tech but I understand it's fairly common in creative fields and--while it was a long time ago--I worked a couple unpaid internships when I was in high school in chemistry labs.
A railroad engineer intern can move cars around the yard all day, assembly trains, and other such things. However if they move the car to a loading dock where it is loaded/unloaded that is valuable work and must be paid. If the train they assembly leaves the yard for the next that is valuable work (unless they all make it back to this yard unused).
Most companies choose to pay interns rather than figure out how to not break the rules. A few have a separate intern program where they carefully figure out tasks to assign interns for training. If your intern program did valuable work you should talk to a lawyer.
Not to mention, you'd have to have a precise and well defined notion of what "analytical hobbies" are. Preferably one that doesn't carry an inherent classist[1] (and maybe even racist or sexist) bias. That part alone seems pretty tough, forget the actual definition part.
---
[0]: https://en.wikipedia.org/wiki/Eight-legged_essay
[1]: https://hbr.org/2016/12/research-how-subtle-class-cues-can-b...
That's the truth for me, I was shocked some of our worst* interns went on to work at Google after the summer.
* Unproductive, slow, needed lots of hand holding
All interns need handholding but some need 10x more than others. And yeah, they often end up doing another internship or so at a faang.
Alternatively maybe those people were just unmotivated to work for you and spent their internship preparing for Google.
At least in our place, we do not really expect them to be productive in the usual sense, but they definitely have ample opportunity to give us a feeling of their place on the bell curve.
(as an anecdote: We currently have some 17 year old apprentice which puts some of our seniors to absolute shame, myself included. The amount of motivation, learning hunger, quick thinking and ability to ask the right questions is really astounding, especially given that he essentially just started and has next-to-no education in our area to speak of as of yet)
Edit: I have ADHD, I couldn't do work for shit in any job, but I could ace every interview. Then I got a diagnosis, got pills and became a star employee. This is absolutely a thing. I don't see why people would downvote this.
Too be clear, I’m not accusing you of being lazy, but your description seems to apply to me and I consider myself lazy.
At this point the whiteboard gauntlet seems more indicative of a lack of critical self evaluation than anything, “we only hire the best because we’re the best rah rah”. Miss me with that shit.
Google's search results aren't because they suck, they're doing it to get maximum money. This is about getting maximum cash for minimum overhead. The Goog is a publicly traded company, they're about selling you the crappiest product at the highest margin. Until someone can come and seriously eat their lunch in search, it will never get better.
To paraphrase Cory Doctorow, "en-shit-ification" is explicitly the point.
They hired the best and the brightest not to make search better, but to find ways to optimally increase the stock price; good products are entirely tangential to that.
Google also operates with their own sets of constraints. What works for them isn't necessarily what works for different types of companies.
Isn't the story more complex than that? I certainly didn't just get leetcode questions when I interviewed there, and when I asked, the interviewer said the group hadn't been happy with hires that only knew algorithms.
2. Did you get an offer? Did you accept?
I had something like an 80% offer rate because of this. Ironically I didn't actually get a Google offer though (not sure why, I did solve the problems).
It's also more common now to have two coding sessions, and even two system design ones, under the rationalization that it reduces interviewer bias.
Google explicitly links to leetcode/hackerrank/etc as suggestions for practice resources in their pre-interview email, implying that practicing artificial DS&A quizzes is now table stakes.
I would be curious as to whether these changes have been data driven, or whether the bar is being increased purely out of necessity to weed out candidates more strictly.
I certainly have seen my share of candidates that do fine in leetcode sessions but then go on to fail in the other sessions for various reasons that allegedly correlate to work performance.
> Google did the research
Did they? Where is the proof? Pot, Kettle, black
But it is hard to say how much leetCode style interviews helped. Post-2015 Google already had quite a bit of momentum.
Cracking the Coding Interview came out in 2008 and had a short section talking about the Google interview process, which I believe was system design + a coding problem that would probably be close to a modern LC question.
I did a fair number of coding interviews (for SRE positions) at Google (I don't actually know how many in total, but 25-30 is probably a safe guess) and, yes, it started with a small problem that should be solvable in well under half the interview time. I only used problems that passed my "is this fair to the candidate" screening (look at problem, try to write a working solution, if that takes more than 7 minutes, the problem is not fair).
The value in the interview comes from (a) listening to the candidate narrating their thought process during the coding, (ii) discussing the solution with the candidate, sometimes in terms of complexity, sometimes in terms of correctness, depending on what is on the board, (3) refining the code written (add more functionality, change functionality).
For a language I knew, I tended to overlook small syntactic mistakes (a whiteboard does not have any syntax highlighting) and if there was any questions about library functions, I would give an answer (and note down what I said, so that would be the standard used when judging) and the bulk of the score came from the discussion about the code, not the code itself.
If that matches what you mean by "LeetCode-style interview", that's certainly been in place since at least 2011. If it does not, things may have changed, but at least the aim wit ha coding interview back when I was still at Google was less "get some code judge, on the code" and more "get some code, judge the discussion about the code". It is also entirely possible that interviewing for SWE positions was different frmo hiring for SRE positions.
Basically the equivalent of the Roman emperor thumbs up/thumbs down except it's a small group and you need a certain number of thumbs up to move forward. The number of people you need Inclined to Hire differs from org to org that use this method.
I worked for Google and at least back then everyone working there could read the studies showing that the interviews worked. The effect was very clear, with there being no upper bar where the interview score stopped mattering.
If you haven't seen any data then you just have a set of anecdotes, so your position is very weak. I have seen the data, but a person like you who is convinced it work wont believe me of course. But I'm not sure why Google would lie about that, or why they would keep the interview format for so long if it didn't work. The old article talking about Google interviews talked about what they did before they arrived at this format, what they do today is the best interview according to their data.
Basically, the same depth of evidence (maybe less) that saturated fat was causing all this heart disease, so let’s keep everyone on very low fat diets.
I just remained unconvinced that we know anything at this point. Google is allowed to do what they’re going to do and to do their own studies and make their own decisions based on that data.
However, that study is not a normal lock for the rest of the world and people SHOULD be questioning it, even with anecdotal evidence, at least till we have enough studies proving their point in different situations and companies.
I don't work at Google, so I cannot speak for them. My company does have studies and experiments around interviewing, but again, the metric they use for success is "inclined rate" e.g. doing ABC leads to x% increase in inclined rate. I haven't seen any study linking interview outcomes with on the job performance e.g. yearly performance rating or promotions. In fact many people have requested this data over the years.
> If you haven't seen any data then you just have a set of anecdotes, so your position is very weak. I have seen the data, but a person like you who is convinced it work wont believe me of course.
This comment seems quite rude, and again, it misrepresents my position. I didn't say LC style interviews don't work, I'm saying that I don't know, because I have seen no data to prove that they work better or worse than other alternatives.
Also the landscape has change, and what might have worked 5 or 10 years might not anymore. Tech interviewing has "professionalized" in a Goodhart's Law-esque fashion. People have become very dedicated to game the process, which might affect the quality of the outcomes. But just to be clear again, I want to see evidence one way or another.
But that is exactly what Google did, they linked the interview results to later performance evaluations and that is the correlation I'm talking about. The better they scored on the interviews the better the performance reviews they got later. The correlation was very stable and didn't just appear at low levels.
Your company might not have done such a study, but Google did.
> Also the landscape has change, and what might have worked 5 or 10 years might not anymore. Tech interviewing has "professionalized" in a Goodhart's Law-esque fashion. People have become very dedicated to game the process, which might affect the quality of the outcomes. But just to be clear again, I want to see evidence one way or another.
That is true, the data I saw is nearing a decade old now, but at least the data back then showed that it worked.
One possible explanation of this is that people who did well in the hiring process were valuable employees. Here's another: people who were good at gaming an evaluation system run by Google were good at gaming an evaluation system run by google.
All these processes become items on a hiring checklist, but to your point, it doesn't guarantee hiring a great engineer. Just one of those tragedies embedded alongside other big co bureaucracy processes.
Note the hiring managers are the ones picking the criteria.
My guess is a dev who passes leetcode but fails to produce is not seen as reflecting upon the hiring manager. It doesn't matter if leetcode correlates or even negatively correlates with the success of the new hire; what would matter is not being able to blame the hiring mgr.
Is high performance making economically valuable contributions? If so, there is huge inherent inequality between the economic value of projects, so the metric is worthless for judging engineering talent.
Is high performance getting good performance reviews? In my experience performance reviews are wrought with favoritism, ageism, and probably other subjectivities that create a very low signal to noise ratio, so that metric is worthless.
Is high performance someone who is good at writing software? There are engineers I know who write very good software but lack the decision making or vision to do so in valuable areas. Thus, another metric is worthless.
The reality is there is no general “high performance”. There are smarter and more motivated people, sure. I think testing for general intelligence, personality type, etc., are valuable. But I also think a lot of high performance is about finding someone who is a good fit for the organization in terms of the individual’s experience and skills and the organization’s needs.
This variance is why it is so hard to assign objective numbers.
Bugs are not created equally.
- assign low effort, high impact, high visibility, low risk projects to the most favorite engineers.
- assign high effort, low impact, low visibility, high risk projects to the least favorite engineers.
Favoritism in toxic orgs is the reward for "loyalty".
Another way is by reorganizing teams, assigning favorites to a team just before the completion of a project. Or assigning undesirable employees to a failed project.
I've heard that claim times and times again.
Often it's from companies believing in "Best Cost IT talent" or whatever the current buzzword is on LinkedIn. Chances are, they never even worked with an SV caliber dev.
We've all seen engineers that are simply running around in circles around others. Just think of John Carmack. If any of these organizations managed to luck out by hiring someone from, let's say Apple or Google, it would most likely change their beliefs that there's no "high performance".
if they are regularly practiced at a company, this means the company puts enough faith in them to overall promote those who deserve it and to decide peoples' and company's fate based on that. Which then makes it quite a decent interview outcome metric.
If on the other hand performance reviews are as bad and worthless as you claim, they should be stopped completely, and everyone running them should be instantly fired as a bunch of ageist favoritist pigs who are scamming the company.
They could have been fooled by randomness, like many "intellectual" types often are.
If I remember correctly, it was actually slightly negativity correlated with work quality because extremely talented competitive programmers would smash the interviews but write competitive programming style code (aka not enterprise quality).[1]
1) https://catonmat.net/programming-competitions-work-performan...
the interview is about the interaction on a CS topic. how do you express yourself, verbally and in code? that's not: can you crank your the highest performance solution in the shortest amount of time.
I can tell you that at our company we definitely dodged candidates here and I was very happy with everyone we hired (but you could call that bias, as I was the hiring manager -- but I also worked withe veryone I hired)
FAANG-like hiring has the question "What info do you get from the 5th interview that you didn't already have after the 2nd?" - for the paid trial, what info do you get by Friday that you didn't have by the end of Monday?
To answer your last question.
On Monday, it was had to fully tell how someone interacted with the team (there was a particular incident where a candidate ended pushing some extremely strong politics on the rest of the team on Wednesday). I was okay with it (I think people should be okay to express opinions), but the rest of the team was not -- and I value the rest of my team (ended up making the decision not to go forward as they weren't comfortable with the hire).
Similarly, when they gave an estimate on Monday for the work they could accomplish on Friday (but failed to do it), depending on their reasoning (did they just grossly miss-estimate, or were there legitimate reasons, are they learning from errors). They had time to keep us updated throughout the week, to readjust, etc. All things that you typically learn with time.
How was it gamed? I wasn't aware of anything. Is this about LLMs being able to solve some of the questions?
Sure you need to understand some basic CS to pass a LC question, but it’s more about recognizing a pattern and having memorized the proper algo for that pattern.
If you were taking an algorithms interview 20 years ago, you’d have to truly know a huge breadth of CS knowledge. But now you can just memorize problems for a few months and never learn the true meaning of “why” a tree is the best data structure in x situation.
It was “gamed” when people learned that you could pass the interviews without truly understanding the material, if you just memorize the rules of the game and can recite them when asked.
It's also not as easy as you describe - just memorizing patterns. Maybe for the easy-medium questions that ChatGPT can solve, but Google stopped asking such questions a decade ago. The algorithm questions that all the tech companies ask today require real problem-solving.
Did they? I remember hearing about a research they did and it wasn't so clear cut to its advantages as a "job success predictor"
oral exam style algorithmic interviews are.
Where would we be without such hiring processes that managed to have such set of genius work on Android.
/s
Everyones approach looks like it works because if everyone is between a 90-110% worker and the team gets along and works together you will get solid results.
Big assumption that performance can be reviewed and scored in some objective way, or if it is that this actually happens. I've seen lots of review processes that pretend to be objective, but absolutely none of them actually were.
Same goes to any subject where a multivariate analysis is almost impossible due to the lack of control over your experiments. I seriously doubt that anybody has any data that could prove anything.
As an interviewer who did this for many companies for a decade, I hire for attitude. I can teach anybody hard skills (software engineering, sql, python, etc.) with enough time but I can't teach attitude.
A person who I can trust because the persistent drive towards goals, can deliver a lot more than a naturally talented programmer with shitty attitude. This is based on my anecdotal data that I accumulated over the years.
Funnily enough this is similar to what the Seal Team figured out about recruiting in a completely different field independently.
That's just part of the process, though. They (and many others) have coding questions, system design questions, and behavioral questions. Taken together, they answer the questions of whether you can write code, design/build/contribute to a coherent larger system, and are someone who can be effective at working with others in a company.
Over time, there's been a massive (IMHO) over-index on the coding portion, since it can usually result in an objectively correct or incorrect answer. With massive numbers of applicants, companies really don't lose out on being over-aggressive with cutting people based on these things. As they tighten their hiring belts, I think this will get worse.
[Citation needed]
(Or you are doing what you say others are doing)
We update and share these points within the team and with the applicants even before they make the first contact with us:
https://stratoflow.com/our-recruitment-and-onboarding-proces...
That means “grinding leetcode and working for a FAANG” (c) r/cscareerquestions.
I am 25+ years in the industry. But if any new college grad asks me for my advice. That’s what I tell them.
I definitely wouldn’t tell them to take a chance of working for any non public company hoping their “equity” may be worth something because they heard that early engineers at Uber struck it rich.
The amount that Big Tech is willing to pay is correlated to the market supply of employees.
But I'm unconvinced that a very profitable company will decide to pay less than it can afford, in order to acquire the best engineering talent it can, when their revenue is directly linked to their engineering products.
The only thing that would make FAANG pay nosedive is if they made an organizational decision that they didn't want to prioritize software development.
So basically, if one of them decided to hire an Eddie Lampert type.
They are not o paying maximum of what they cam afford, they are making huge profits.
If they are not paying maximum that they can afford what are they paying? Provavly the minimum they think they can get away with.
Its all mind games.
Relative maximums for profit and talent willing to accept those salaries.
I’ll know myself in a couple of months…
[1] https://interviewing.io/blog/2022-layoffs-engineers-vs-other...
My story is that in 2020, I was 46 years old living in the 3200 square foot house I had built in 2016 making $150K (and getting offers locally for $170k) my wife was also working part time making around $25K in the school system.
We had more than “enough”. We went on two or three trips a year, had “date nights”, saving “enough” for retirement. I ignored every message from recruiters at BigTech. I actively didn’t want to work at any large company as a software developer nor did I want to move.
The only reason that the recruiter from Amazon Retail even piqued my interest was that when she suggested I do a slight pivot to “enterprise application modernization” cloud consulting.
The extra money is nice. But it really just ended up going into my bank account and didn’t make an appreciable difference in our lives besides “retiring my wife”.
I would have no problem going back to my (inflation adjusted) prior compensation.
https://bemorewithless.com/the-story-of-the-mexican-fisherma...
I know plenty of developers 40+ who would never give up their lives in the burbs of Atlanta (where I lived until this year) to move to the west coast.
Even now, I’m almost sure that the new college grads who I work with (and one that I mentored as an intern) that came in after I did at an L4 will be promoted to an L6 long before I will (if I ever get promoted). I actually told my manager and my skip manager that I don’t want to be an L6 or the responsibilities it entails. I’m already saving/investing every penny of my RSUs. My base salary is about the same as it was before I left “enterprise development”. My fixed expenses are actually lower
Heh Heh Heh
Wonder if your wife would look at her being retired as an appreciable difference? :D
By 2020 she was working for the school system part time on the school schedule.
I used the two year prorated signing bonus to pay off all of our debts, increase savings and reduce our expenses.
We also moved from the big house in the burbs of Atlanta to a smaller condo in Florida and we don’t pay state income taxes.
So yeah over the past three years I both increased my compensation and reduced our fixed expenses.
My sample is myself and all my peers.
Industry is BIG and the demand for smart people with a track record for getting things done is even bigger.
Most years I barely do anything more than basic eng management and my teams build glorified CRUD apps.
I'm paid much more than others in the UK because I'm a sure bet. FAANG not required.
In any case, even in that light, the claim of FAANG or don't be an engineer is absurd. Even if someone doesn't end up with a really well paying job it is still a solid career option "especially in this economy"
You just have to play the game right. Excellent developers are rare in the UK in my experience or hiring. Mediocre ones who think they're excellent are everywhere.
> Mediocre ones who think they're excellent are everywhere.
Absolutely
Out of curiosity, what kind of non-FAANG domain are you in that pays soo good in the UK? IB? HFT? ML?
I'm asking since every single UK person on HN complains that UK tech wages are shit yet you claim otherwise.
What makes you the exception or why do you think the others would be wrong?
Thanks in advance.
If they really mean US FAANG total compensation, my guess is quant trading firms.
That's the comparison sirsinsalot was making.
The salary curve in the UK has a long tail of rubbish pay because that's what people accept. Budgets are often much higher, and supply of skills is low.
You just have to follow the money and negotiate well. Firms with deep pockets don't care if you cost 90k or 300k when the project is in the hundreds of millions and the fallout of failure would cost much more.
The important think is to show you represent 5x less risk for 5x more pay.
There are plenty of us making US FAANG salaries outside the US and not at any major tech company.
You can have pretty good salary in non-FAANGs, make a difference, and match your own software values with what you're doing.
I would even argue that by doing so, you're elevating those company, allowing them to then compete a bit more closer to FAANGs comps, and "stealing your talent" away from those less-matching software values, creating what I believe (from my own values) to be a virtuous circle.
Or you can grind at FAANGs, accept stack ranking and elevated salary, play the yearly promotion game and end up with a very nice pile of money. Then clearly do not expect them to change, and I would argue: you do not get to complain unless you are actively trying to change them from the inside
And let’s not pretend that startups are trying to compete with BigTech. They are just trying to survive long enough to get acquired. Out of all of the companies that YC has founded, how many have gone public?
If you are working for a company that has accepted VC money, your “values” don’t amount to much at all. The only values that matter are those of your investor and all they care about is an exit - statistically by getting acquired.
The opposite choice is accepting Monopoly money - “equity”.
True my unvested RSUs are half what they were at their highs. But at least I can sell them and trade them for real money once they vest every six months.
Don’t get me wrong, I spent my entire career from 1996-2020 working for mostly unknown companies working as an “enterprise developer”.
The only reason I fell into my current BigTech job is because I both know how to talk to customers and I can create pretty diagrams, PowerPoint slides and a shit ton of yaml and HCL and can develop (cloud consulting department)
The second part is that I tell them to save aggressively and diversify themselves by selling their RSUs as soon as they vest. They wouldn’t use 25% of their cash salary to buy their company stock, why keep your shares once they vest?
I sell all of my RSUs within the six months after they vest and diversify
I would much rather to have loved and to have lost than to never have loved at all. I used every penny of the after tax difference between my “enterprise job” I had in 2020 and my BigTech job that I got then to “increase my net worth”. If I lost my job , it would have been a good three years.
As I said in another reply, I could afford to ignore BigTech recruiters at 45 making about as much as a returning intern makes where I work now. I already had the big house in the burbs, retirement savings, a family and a son graduating from high school.
I would never tell a new college grad to ignore the same recruiters I did.
I could and can take jobs I enjoy, actively run away from promotions, etc.
I had a great time, and learnt the most, at smaller places and lower paying jobs. Then I also had a great time at some better paying startups. I probably enjoyed my "big tech" better paying jobs the least. All that said, you're generally so much above the average as a software developer that you can't even imagine what most people are struggling with.
YMMV
When I was still in college, most kids were graduating with debt. Tuition and fees have only gone up since I left. Lots of people I knew who dropped out to try and save for college ended up in dead-end despair jobs.
Does nobody in the US live with their parents while going to university?
It's not uncommon where I live for parents to save to pay for their children's education. Believe it or not, the government even gives you money if you do that.
EDIT: That said, the US has plenty of good paying software jobs. You can pay back your loans even if you don't work for Google. I'm sure this isn't easy for everyone (especially these days).
The tuition fees are probably a bigger expense than your living expenses if you go to college in USA.
Please tell me how a full-time student can fairly easily earn 10 grand a year (plus books, clothing, phone bill, entertainment, laptop, etc)?
In 1996, I graduated from an unknown state college in South Georgia and I made $11/hour as a computer operator working on DEC VAX and Stratus VOS mainframes.
Because of…poor career choices …I didn’t hit six figures until I was 40. Even now I make about what a software engineer II makes at my same BigTech company at 49 in cloud consulting.
As I say, please play the worlds smallest fiddle for me.
Last year, my wife and I got rid of everything we owned that wouldn’t fit in 3 suitcases - including our cars - and bought a vacation/investment property in a resort area in Florida. We stay here half the year from October - mid March and we fly around the US the other half of the year “digital nomadding”. My wife in the meantime is retired at 47 and she is involved with her passion in the fitness/wellness industry and meets up with people everywhere we go and flies to conferences from wherever we are staying in a given week. I also fly for work a few times a week.
We take Uber everywhere.
This is what I meant by I can make different life choices at 49 than I would recommend a college grad making. I can be okay barely making L5 compensation.
Our fixed expenses are really low.
I'm not quite sure what your point is. You seem to be supporting what I'm saying which is you can get by perfectly well with various choices. You made choices (poor or not) to not go after the biggest paying jobs and you're doing great. Why does a new grad today have to go work for Google?
In the US there are lots of really good paying software jobs that aren't with the biggest tech companies. I've worked remotely for a small US startup many years ago and got paid really well. The current market is tight but over the last decade if you had a pulse and could code you could get pretty decent job.
I also had no college debt (scholarships and going to a cheap school) and I was able to get into BigTech by being old, with industry experience that allowed me to bypass the leetcode grind. But only by pivoting into consulting.
These are life choices I could make that a new college grad couldn’t. Also when I graduated, $BigTech wasn’t a thing. Apple was barely hanging on for dear life and MS wasn’t paying more than your average company adjusted for cost of living.
That’s because you have enough of it now.
That's not to say that one should work in oppressive conditions or make other people suffer for a little more money; there is nuance. But, seriously. You can't pretend that money doesn't matter.
You're assuming that everyone's goal in life is to be happy as defined by material comfort or hedonism but that's just not the case. There are many non-hedonistic things that are enabled by having money (philanthropy and venture capitalism of various forms, freedom of various forms, sense of security, etc.)
For such people, because hedonistic happiness is a worthless commodity, trading it for money, and by extension all the non-hedonistic things that money can buy, is a very favorable transaction.
It's easy to see why earning more money by sacrificing comfort doesn't make sense to someone who considers happiness as their goal in life -- it's a longer route to reach the same point.
Don't assume things and don't comment here with bland reddit one liners. Only say something if it's constructive.
The vast majority of people I’ve hired want to be happy, and I have completely qualitative conclusions from having hired such developers for decades: It works well for everyone.
If you want a job at FAANG badly enough to suffer it, go for it. But don’t ask me validate “that’s what everyone is doing”. It’s not — you’ve been sold a bill of goods, or you’re greedy. Either way, you be you.
But I bet you dollars to doughnuts that if I told any of them that they could double their compensation by “grinding LeetCode” they would trade the “equity” in a private company that statistically won’t amount to anything for real stock being in their brokerage account every six months.
As far as being “greedy”. Are you running a charity?
Students might know starting salaries, but maybe not know just how huge the income difference can get in a few years. Nor understand how adversely the relative difference can affect their quality of life.
> If you want a job at FAANG badly enough to suffer it,
I mean, you have to agree, for a lot (if not most) people, the ROI is massive.
You can't possibly know that for sure. Are you making that judgment based on what candidates tell you during interviews? Because the meta-game of a job interview is literally just figuring out what the interviewer wants to hear and then saying it.
At this point, the cargo cult of Leetcode is often more of a hinderance to startups. Everyone wanted to follow Google thinking it’s the best system without asking themselves if they have the same problems or are in need of those kinds of people.
Second approach is just not scalable in the long run. And you need someone to do grunt work.
If your premise is "We look a lot like Google circa 2015, and I really like what they've done since then, how did they do it?" then it might make sense to replicate their hiring.
But if you want to achieve what Google achieved between 1998 (founding) and 2004 (IPO) then asking what they do now (or have done over the last 10 years) isn't particularly relevant.
If the comeback remark is something like, "if you really want this job," I will reply, "if you really want to hire me."
My current job, I told them that I was too busy to do such a thing (and I was), and got hired anyway.
Nobody w/ actual responsibilities in their life should be coerced into doing something for free, for someone they do not know.
Will an attorney give you free legal advice until you decide they are fit to represent you, or will you get free surgery until someone proves they won't completely butcher you? Can you drive a car for free (for 5 - 8 hours) until you decide that's the car you want to buy?
Why is it any different in the software industry? Because we just clack on our keyboards all day and do nothing?
I agree with your point overall, but I do have to say that it is very common for lawyers to give free consultations with potential clients, often offering very useful advice. This is something I've benefited from more than once, actually.
These are bad analogies because both have extremely extensive tests that are not only unpaid but the tested pays a small fortune for.
If development had the same no one would be asking you.
>I will reply, "if you really want to hire me."
It’s not about hiring you it’s about trying to prevent hiring the wrong person which is extremely expensive in time and money and takes weeks to figure out.
I've passed tech interview challenges only to fail the culture fit because I thought it would "be so easy" after the tech challenges.
And I've passed culture fits only to fail the homework assignments.
In one circumstance, not realizing that a separate recruiter was sending me to a company I had previously interviewed with, I've also seen previous work that I did in a homework assignment, given to me in a different homework assignment with a "what improvements and features would you add to this solution?," when the original homework assignment was the same task. That was a major blow, and gave me feelings they were using some of that work internally.
I get that it's extremely expensive in time and money to hire the wrong person.
On the other end, it can also be extremely expensive in time and money to not be extremely selective of whom you want to work for.
This is a perfect analogy.
I went to a Chevy dealer once, asked if I could test drive a car and they wouldn't even let me take it off the lot. I was allowed to trundle around the rows of cars at walking speed with a salesman in the passenger seat.
I went to a Honda dealer, and they let me take a test drive with a chaperone, but only around a short designated loop of streets "for insurance reasons".
I went to a Mazda dealer, and the salesman said he was busy and tossed me the keys and said have fun.
An acquaintance went to a Subaru dealer, and took the car they were considering home overnight.
You a Chevy.
I think it's more up to the individual dealer than a policy of a given brand
Some years ago, a Nissan dealer offered to let me take a high-end crossover home for the night without any prompting. Ended up buying it. Would buy again from that dealer if I were still in that area.
Gestures of goodwill go a long way in sales, that's why people love to pay for coffee or lunch when out with potential customers.
All but one new legal engagement I’ve entered into started with a free consultation. (The only one that didn’t was a straightforward real estate transaction where I knew the lawyer for years beforehand.)
The proceedings to jettison a bad hire can take 12 months alone, meanwhile they're still a bad hire being paid.
And if you cant tell whether they are good developer after 3 months, how is an intervoew going to help?
Rightly so, it should be hard to fire people and should require first showing evidence of trying to work together and help in good faith.
No, it is because some people have worked at very impressive jobs and probably just clack on keyboards all day and actually did nothing while there, so when you hire them you learn that they actually can't do the job you hired them for. You just can not rely on the CV alone.
In the hiring process there has to be some kind of skill test. If it is not a "homework" then it has to be a whiteboard/live coding/system design type of interview and there are a ton of problems with this type of skill test as well.
We usually give people the choice which route the candidate would like to do. Take a 1 hour interview or a "homework" assignment which took me about 1 hour to solve. Which is probably best because some people really prefer the homework because they get nervous in interviews.
> Will an attorney give you free legal advice until you decide they are fit to represent you, or will you get free surgery until someone proves they won't completely butcher you?
I do not know how attorneys or surgeons are hired, but my guess is that the education side of these jobs lines up much closer what is actually needed to do the job (which isn't really the case with Computer Science) and that when they claim to have done X or Y then it was actually them in court or at the operation table and not someone else from their team.
Clients select their attorney based on reputation of the attorney or the law firm they work for. For Surgeons it is probably similar. If you have the choice you want to go to the hospital with the best reputation. If the company you apply to already knew your name and reputation beforehand then the skill test is probably also not necessary, but that is not usual in my experience.
It also probably helps that both of those jobs require a Professional License to practice these jobs. Maybe if the software industry introduced a Bar Exam then these "homework" assignments would not be needed anymore.
> Can you drive a car for free (for 5 - 8 hours) until you decide that's the car you want to buy?
Usually you can. At least in my experience. Not fo 5-8 hours of course, but long enough to know.
If you know of a better way to hire people: I would be happy to listen.
W/ an interactive session, you get instant feedback (either verbally or via emotional cues) into what they are expecting.
With a homework assignment, it's hard to determine which path to optimize for.
If a homework assignment is necessary, it would be better if that were more of a "probationary employment" type scenario, perhaps at a very reduced amount of pay, to only imply that both parties actually have skin in the game.
Even better, for helping me judge if it's a decent fit? Show me some code you're using in production so that I can code review it on the call. Surely it's not all hyper sensitive.
> interactive session, you get instant feedback (either verbally or via emotional cues) into what they are expecting.
For some people, like myself, that is a nightmare scenario. I do not do well in these type of sessions. Which are also not even close to reflecting the real work we do. There are absolutely no meetings I go into that I can't prepare for [1] and there are absolutely no meetings where I have to solve a problem on the spot.
Being on the autism spectrum also means "emotional cues" are pretty lost on me.
There is also the fact that interviews have to be during normal working hours. Some people prefer to do a "homework" assignment which they can do in an evening or weekend.
This is again why we provide the choice. Not everyone is the same and prefers different interview and assessment routes. Both types are useful for different people.
> If a homework assignment is necessary, it would be better if that were more of a "probationary employment" type scenario, perhaps at a very reduced amount of pay, to only imply that both parties actually have skin in the game.
Really? You would refuse a "homework" assignment, but would agree to "probationary employment"? The later just sounds like WAY more work on both sides.
We have discussed something like this in the company I work for, but came to the conclusion that it just is way too much work. Getting the contracts in place and working out the insurance and tax implications and all that. It just is way too much work to do legally, because it would be the same amount of work to just hire them. However: we can't just hire everyone.
Maybe if you are already a freelancer beforehand then we could work something out that way, but in the jurisdiction I am in not every software engineer is ready to accept freelance contracts. It is a simpleish process to do, but not everyone does and those that do don't apply for full time positions.
[1] You can "prepare" for interviews, but more in a scatter shot approach studying all the interview questions that could possibly be asked. That is not what I mean. There is no meeting I am going into where I do not know the precise topic that the meeting is about.
However, I've also been a few hours into a homework assignment thinking that I could probably go down some rabbit-hole to try to perfect something that I may have been struggling with, and sometimes can't determine the appropriate stopping point.
The "probationary employment" would be more like, I don't know, a gift card, or something, vs. something formal.
That way, if I totally bombed out in some assessment, no big deal, here's something for taking the time to apply, and maybe I could use the card to buy a book.
Now, I get that companies aren't giving gift cards away to all of their interviewees, so this type of thing would only come after at least the first round of interviews, etc.
More often than not, a non-interested company will often not even tell why they didn't pass the assessment, and it generally feels like a waste of time.
I had someone try to implement a whole relational database when the interview task was just to read from a CSV file and provide a REST API to the contents of said file using any tech stack. Impressive for sure, but unnecessary time wasted.
> The "probationary employment" would be more like, I don't know, a gift card, or something, vs. something formal.
We discussed something like that in the company. Mainly because someone asked to be paid for the time spent on the "homework" assignment. We came to the conclusion that there is no real legal way for us to do so. We probably spent more money on discussing the possibility of paying the candidate then what 1 day of work would have cost us, but the cost wasn't even the issue.
With an in person interview there was maybe a chance. Inviting the candidate out to the exchange, giving a tour of the trading floor and then paying for transportation, lunch, dinner and hotel would be no problem.
Though interviews are online now as we are a "remote first" company anyways.
We just can't pay for work without a contract, insurance, tax and background checks in place. We can however ask them to complete a test. Which is what the "homework" assignment is.
> More often than not, a non-interested company will often not even tell why they didn't pass the assessment, and it generally feels like a waste of time.
It sucks. It just is that nothing positive can come from providing feedback and you open yourself up for a lawsuit.
Another thing is, if they let you go after the probationary period, you pretty much start the search anew. If you applied to another company while you were 'employed' by X and you managed to schedule another probationary period with Y right after the end of the trial period with X, you have to refuse them last second in case X wants to hire you. If you didn't search for anything while at X, you're off work for another week/month while you search for another trial.
If X ends up giving you an offer and you want to consider Y, what are you gonna tell them? Please wait a month for me while I work for this other company and see if it's any better than you? This simply doesn't work, for both companies and candidates.
Your proposed solution works, if X is your dream company and you will accept their offer no matter what. That's not how most job searches go though.
For me, I work on a lot of open-source as side projects, so there's always the coding "something" factor.
No there doesn’t. There isn’t one for the CEO is there?
I do not know how it works with CEO positions, but I am not hiring CEOs. I do hire software engineers and would like to work with competent people.
If you know of a better way then I would like to hear it.
We can't hire everybody. I can't hire 20 people (then fire 19 of them) for the 1 position in the 8 Person team I want to fill. That can't work.
Just hiring someone and then firing them 2 weeks later is expensive as fuck. We can't just do that until we find someone actually competent.
Your "easy solution" is only easy if you don't think about it at all.
I'm not saying you don't do ANYTHING to hire someone. Obviously there needs to be some judgement of skill. But take home projects where you're judged essentially against how much time you put into them (I've personally had projects where they asked me to build an entire web app, it's insane) because ... it's a race to the top in a way here, if you spend more time it's going to look good, because the best projects are from people that spend a lot of time on them...
It's just ridiculous. You end up doing 8-16 or more hours of work for a job you likely won't get (it's free for you to ask me to do this test, so you ask everyone to do it).
It's not expensive to hire and fire someone. Not that expensive. It's probably worth 20hrs of 100 candidate's time, that's for sure. If you don't think so, I have a bad opinion of you.
I'd like to ask you, how many candidates a year do you have turning in take home projects?
We align the Take home assignment very close with what the position actually entails. If you struggle with this assignment you would not succeed with actual tasks we do daily.
> Obviously there needs to be some judgement of skill. But take home projects where you're judged essentially against how much time you put into them
As I said elsewhere as well: People are free to choose to do a Technical Interview instead. If you fear that you would need 2 whole days you could take the 1 hour interview instead.
> if you spend more time it's going to look good, because the best projects are from people that spend a lot of time on them...
That is not my experience. There are people that just can't make it look good.
> It's not expensive to hire and fire someone. Not that expensive.
It absolutely is. Maybe not where you are, but where we are it is expensive. For one there is a big amount of paperwork involved, contracts, NDA's and background checks. We can't do this unless we know we actually really want to hire you.
I also can't expect a new employee to be productive right away. There is going to be a onboarding period of at least 3 months during which the employee probably needs a little bit more mentoring of a (more) senior engineer. If we would just hire 20 people for every position that we need to fill we would not do anything else then just onboard new people. So, YES, it is significantly more then 20h of 100 candidates time. Not to mention that the take home we give out takes maybe at a maximum 4h (if the candidate is fairly junior and has to look up stuff constantly, many people just do it in 1h).
Also: I think it would be HIGHLY unfair for the people to just hire 20 candidates if you only plan to keep 1. They probably quit their jobs or said no to other opportunities. We only hire people we see a long term future at the company.
> I'd like to ask you, how many candidates a year do you have turning in take home projects?
It is fairly late in our hiring process and it takes a significant amount of time from at least 2 senior engineers each time to review. I don't have the exact numbers how many times per Year, but we try to keep it very low and only if there is an actual interest on our side.
Ah, I missed this, and I'm sorry. My experience has not been this over 100+ technical interviews over the last 10yrs.
Almost all of them have asked for crazy take-home projects. I'm pretty bitter about the fucked process.
> In the hiring process there has to be some kind of skill test
Thats a contradiction:
1 - 90% of companies do skill tests
2 - you cannot trust the CV of a candidate even if the company they worked on previously also did a skill test
3 - you think your skill test will enable you to hire the right candidate
So everyone is doing tests and everyone is hiring shit candidates anyway?
That's literally proof that these tests are worthless.
Maybe if the recruiters actually knew what they need from a candidate, companies were clearer about what the job involves, they would stop hiring the wrong people for the wrong job.
Most lying is apparently to change start and end dates of previous employment to cover up short stints where they may have been fired for poor performance, but even if work samples/skill tests/structured interviews were 95% effective, you would still regularly hire duds.
We had one person who literally hired a team in India to do it for him. Presumably planning to give his work to them later if he was hired.
You sound so jaded, I don't blame you, but I hope you're not ever my hiring manager.
Needless to say we didn't proceed with signing the contract.
2 - We can not trust the CV. Maybe the company did a test, maybe they didn’t. Maybe their test are to our standard, maybe they aren’t. Maybe the candidate barely passed. Maybe the candidate actually failed but made up in other ways that might be relevant to the other company, but not ours. I doubt anyone would give us this information about their current or former employees. Maybe the candidate really did work on the projects he claimed or maybe he was only tangentially involved. That is a lot of unknowns. A test clears up what the person actually can do.
3- Yes. It works for us. Do we miss out on some good candidates because of this? Yes, but it definitely prevented us from hiring bad candidates.
Again: if there is a better way then I would like to know. I have not found one yet.
There is a large company based in NYC that interviews thousands of developers a year, but only hires a few hundred. Each of those devs do a take home project that takes about 16 hours to complete. The project requires nothing to ask, so this company asks everyone. 30,000 hours is a lot of time - it's about 3.5 years of someone's life - and at least this time is spent each year by this company (and they pay nothing for it), not counting the rest of the interview process, because it is cheap for them to do this.
And we wonder by productivity is down :)
Respect people's time. If it is hard, figure it out. That is your job if you want to hire people and feel good about it.
Will an attorney give you free legal advice until you
decide they are fit to represent you, or will you get
free surgery until someone proves they won't completely
butcher you?
If you do that in software, you're probably just back on the job market...
I have my own list of questions. If they answer all of them I have a pretty good idea if I'm the guy for the job. The most wonderful part is figuring out if it is an employer is looking for initiative or obedience. If you are running a sheep farm it can be very exciting to see initiative.
IME you also often get paid (albeit a relatively small amount) for doing them. And FWIW once they want to hire you many employers will actually spend hours trying to sell you on joining. Usually not in the form of an essay, but doing several hours of sell calls or lunch meetings with different people at the company is not that strange (especially if you are higher level).
That being said the come back makes perfect sense to me. When I was still on the IC track and interviewing, when I was submitted to some "dumb" questions I would ask the same kind to the interviewer. So you just asked me to write this stupid algorithm? How about you whiteboard something for me now?
It was almost always met with dead silence to which I would say "well you're trying to figure out if I'm good, I want to do the same and make sure your team has good engineers".
Of course it never went anywhere since, while they had 55 minutes to grill me I was only given the last 5 for my questions.
While there are people who are naturally great at interviewing in person or who don't mind grinding leetcode for free for months on end, there is a whole "base of the iceberg" population who a) can't spend months grinding leetcode on end but they can definitely spare a couple of afternoons for one job they are interviewing for – these are normal people with normal jobs and a normal family who don't interview every month just for kicks, they do this a couple of times every 2-3 years, and, b) just cannot code while under stress. You may scoff at b) if you've never experienced it, and while I found it really hard to understand for a while as I also have no issues with getting into deep focus while people are looking and/or trying to talk to me and there's a time limit, it is absolutely real.
Unfortunately, there is an extremely vocal minority (you) who go absolutely ballistic when asked to do a take-home exercise, who absolutely ruin it for everyone else and make hiring managers shit their pants whenever take-home exercises are suggested. I honestly don't understand why there's such an outrage, take-home exercises are the minority already, because there's a strange huge backlash.
Sure, if you're a great communicator (usually native English speaker) and grind leetcode for fun so you can shove 5 interviews in one week, you hate take-home exercises. And that's fine, really, just please apply to the leetcode grinding contests and stop poisoning the well for the rest of us who would like to cater to the large amount of extremely competent engineers that don't fit that persona.
> Nobody w/ actual responsibilities in their life should be coerced into doing something for free, for someone they do not know.
I take it you haven't interviewed for a SE in a while? The status quo requires you to perform months of unpaid labor, not days, just in the form of memorizing the solutions to every single Leetcode Hard problem. How is that better?
I guess because it kinds "scales". Once you are familiar with Leetcode you can use this skill to take interviews with different companies. Take-home exercises dont' "scale".
That being said, I believe people are against take-home exercises exactly for the reason you support it: it's usable code. They worry the companies will exploit candidates by using their code for free. Leetcode is "useless" so it's safe.
If we're talking about "make me a website" kind of exercises that one could throw into production after some tweaks, then that is free labor and I would absolutely refuse those.
Also, the fact that "leetcode scales" is part of the problem I was talking about. I really dislike the fact that if you train for months to develop a complete set of skills parallel and irrelevant to your actual role you can now efficiently interview for a number of companies that pay top dollar. So not only people with actual skills and experience are on equal (or worse!) footing than someone straight out of college, it also incentivises those who mastered leetcode to interview everywhere, since "it's all the same", while normal people who are good at their job get their torn to shreds after one or two interviews. So not only it's judging the wrong thing at the interview, but it's also causing a starvation situation for the candidates that are not playing that game.
Why are we creating incentives to people becoming "professional interviewers"? This would be like Google encouraging SEO spam instead of fighting against it!
With recent layoffs and many talented professionals on the job market, I was compelled to write a blog post about how to build an inclusive hiring culture and find exceptional engineering talent.
If you're involved in your organization's technical hiring process at any stage, I encourage you to give this a read. I share some best practices for conducting effective interviews and improving your own hiring process.
Let me know what you think!
Thats a contradiction in terms. Building something exceptional always involves excluding mediocrity
You can exclude mediocrity while also being exclusionary on other axes.
They are unrelated issues. In fact, I’ve even heard of exceptional people being abused/bullied for belonging to the wrong group to the point of being told “you couldn’t have done that” which itself is an assertion of their supposed mediocrity for exclusionary reasons.
Don’t assume you need to be bigoted to exclude mediocrity. Discriminating, yes, but not discriminatory against groups that inclusive hiring policies attempt to protect.
An inclusive interviewing process does not mean that you hire everyone. It means you reduce the weight of people's biases as part of identifying who you hire (because people turn out to be quite bad at prediction in hiring).
This leads to organizational pressure to hire based on population distribution. Doing so inevitably means hiring based on attributes other than skill.
- The companies that make an effort get a lot closer to population baselines than the ones that just give up.
- Organizational pressures are something leadership and management should be steering. I'd rather have hiring practices be an explicit choice than something that "just happens"
If your goal is population distribution then you are inevitably hiring based on attributes other than skill.
For example, women don’t make up 50% of the engineering talent pool so if your goal is 50% women then you have to lower standards to achieve that.
You don't have to. But you'd remove them from the job market, making the pool smaller for other companies. IOW, a few companies can target 50% women, but that'll make it that much harder for other companies.
Let's say you are successful. Let's say a class of companies are successful at this strategy. Lets use the example of faangs which are desirable places in terms of salary/brand. If faangs were successful at this that would reduce the % of female developers in other industry and assuming faangs are taking the best candidates that leaves the worst ones. Which then creates this reality where male programmers outclass female developers in these other industries. That makes it harder for women in general and makes this false impression that females are not as good as males.
To help women you really need to treat them equally. Trying to reach a goal of unhealthy unnatural % industry wide means women will left holding the bag when the music stops.
But this also creates a positive feedback loop when more and more women decide to switch industries and pursue IT/Engineering jobs via bootcamps, college degrees, etc. I noticed the number of female candidates in UX/UI, fullstack, QA, Data Analytics - has increased in last several years.
Partly because the demand is still high for these professionals, partly because there is entire cottage industry of bootcamps churning out IT specialists en masse, partly these diversity hiring practices that opened up doors for women
They churn out ego inflated beginners who think they're experts.
The mentality peddled by bootcamps to sell their wares produces dangers "engineers" in my opinion.
I have no formal education, I'm not coming from educational elitism here.
I'm a bootcamp grad, and would not have gotten into the field if bootcamps did not exist. I'm about four years into my career now, currently working at a major well-reputed tech company, and haven't gotten an average-or-below annual performance review yet. (And one reason for that is that I tend to be cautious, critical, and thoughtful in my technical decisions.) There are a number of other people from my bootcamp class with similar results.
80%+ of the bootcampers were rubbish and shocked to be told their knowledge was way below where they thought it was.
A classic is a 6 week JavaScript bootcamp grad claiming to be an "expert in JavaScript" (their words) and couldn't explain the JS type system or basics of variable scope. That was the norm. That kind of rubbish.
I'm happy you're an exception and everyone gets a fair chance with me, regardless of background, but I am never shocked when I have to bin yet another bootcampers CV
If you mean "in my experience, bootcampers fail technical screens at much higher rates", then say that, instead of implying that you're stupid if you even consider hiring someone who went to a bootcamp.
If I could interview everyone I would, and I'd happily hire a bootcamper that seemed excellent.
The reality is that statistical likelihood of passing screening means those CVs often hit the bottom of the pile.
The two statements I made aren't contradictory
We're always hiring based on attributes other than skill. If we're lucky and purposeful, skill becomes a part of the hiring process.
I hate when companies lie about that, it massively lowers my respect for them.
At least this is meritocracy, the kind of thing that people (e.g. eugenicists) can make a serious argument for.
> look like me, speak like me
...is something that can't be justified except by terrible people. Even worse is "likes the same music and movies that I do" or "we coincidentally have mutual friends."
I have bad news for you about the people commenting in this thread.
This is wrong by definition
US army was exceptional in 1970's, did you need to graduate from harvard to join? No, they drafted everyone, even your sorry ass didn't want to join.
British industrialisation was exceptional, they didn't exclude anyone, even got children something to do by sending them into coal mines!
Amazon is exceptional, is it hard to become a worker in Amazon warehouse?
Organisation can be exceptional without any individual being exceptional.
Also you could be exceptionally bad!
You are moving the goalpost from mediocrity to disability.
It's illegal to enlist anyone with an IQ below 80 to the military.
They don't have any use for them.
Approximately 1 in 10 people have an IQ below 80.
As soon as people figure out the check boxes or the structured pointing system they start to check all the boxes, but it doesn't necessarily speak to the nuances between individuals that make them diverse and both valuable. In fact it can lead to a certain type of person people hired or promote, which can be on good characteristics, but I find many times turns into a "certain type" of person.
I guess what I'm saying is structure can take you so far, but you have to be willing to explore a little bit about what makes a person special, and that many times means not controlling the whole interview, and be willing to have your bias challenged through the candidate directing some of it.
I think checklist interviews miss the mark as you say. You may not have a perfect rubric, but I’m not grading students on a history exam; I am evaluating them for a role in a given position. In the limited time we have to speak, I want to use my intuition and experience as an interviewer and engineer to rapidly get to where the candidate is strong, and where they may have issues.
Does it sting when I get a "No"? Yes, a little, but I did my best and (presumably) someone else did better. So, I take solace in that I did not have to make a (relatively large) decision.
How the heck do you measure actual, repeatable performance? Or skill? Income / promotions / etc is very frequently a horrifically biased metric, for similar reasons to interviews, and we have much larger mountains of evidence showing that to be the case. It seems like there's a pretty good chance these studies are just measuring relative bias between interviewers and the interviewee's management, and concluding interviews are done poorly when they disagree with management. i.e. "structured interviews force people to think more like managers" rather than "structured interviews more accurately measure skill".
Using one bad measuring tool to conclude another tool is bad seems... problematic at best. I will grant that "interviews should measure what managers measure" is often what businesses want in bulk, but that does not seem like a particularly good thing to me.
I once spent 2 hours coding Tetris for an interview. I lost to another candidate that completed 2 more features than I did in the same time period.
Been treated in past like I'm avoiding the "important part" of solving their quiz and that it's some softball topic.
You want to see repeatable behavior and a general interest in going through the process. If someone takes the time to apply with homework and is able to articulate well, it gives you so much valuable signal.
Pressure cooker style interviews only reveal someone can remain focused under stress and that they studied their leetcodes.
Also, stop giving take home projects. Bad candidates will cheat them and good candidates will not even do them. If one of the random startup names listed on the author's site sent me a 12 hour take home project I would delete the email. Do you think they pay twice as much as the bigger company that only makes you waste 6 hours doing a whiteboard? I doubt it.
And I think that smaller companies copy this as a part of the tendency to copy large companies without thinking about whether the thing they are copying actually makes sense at their scale. In this case it can be very damaging, because false negative for a startup with a limited pipeline can be very bad.
You're right about the copycat behavior. This goes all the way to top of funnel: these small, even trivial-scale web application startups just don't have hard engineering problems. Many imagine they do, or imagine they will once they take off, but the work they're offering these high powered candidates they claim to want to hire is like, wiring up CRUD apps and making javascript buttons. It's not technically deep work, it's product work. A little humility about whether or not your tech startup is truly doing "tech" problems would, I think, fix some of the expectation/reality mismatch people are having when they complain about how hard it is to hire engineers.
And sure, lots of people join Google to work on world-scale problems and end up wiring CRUD stuff anyway. But they can at least plausibly offer some technical depth (or could anyway, perhaps Google's reputation as a great place to develop an engineering career has been fading).
(this is not to knock on "CRUD" but to highlight that a technical problem solver is an overlapping but not identical skillset to someone who can work with a fast moving team to quickly and reliably develop product changes)
Yes, yes, 1000x yes. I was recently rejected after two technical interviews from a company that my former principal engineer that I worked directly under had referred me to. The position was to work under them again, which is why they referred me. The feedback I got from the recruiter was that it wasn’t the result they had expected, and I hadn’t achieved a specific number on their technical assessment. My understanding of this after some discussion was that the lower score was due to my speed in answering the leetcode style questions in the live interview with another engineer.
Here’s the secret I never brought up with the company while interviewing: In high, school, college, and for the CPA exam, I received accommodations for extended time and testing in isolation to reduce distractions from my ADHD. With those accommodations bringing me up to an even playing field with a neurotypical test taker, I was able to get into a good university, graduate with a bachelors and masters at the top of my class, and pass all four sections of the CPA exam on my first attempt. In the real working world, I have never needed extended time. I always deliver what is asked of me on time while I have witnessed neurotypicals show up to meetings with their work majorly behind.
I have always hesitated to bring this up with companies because I fear they will make the incorrect assumption that extended time on testing implicates that I will be a slower worker, which I have not found to be the case. I don’t want to introduce any biases for the interviewer to pick up. For whatever reason, testing with pressures absolutely slows down my thinking. In the real world, I have found when I face particularly tough problems, I find solutions after going on a 15 minute walk outside or while taking a shower in the morning. You cannot test for that style of problem solving in these high intensity algorithm technical interviews.
I certainly miss having a CPA license as evidence that I was a competent individual from my previous accounting career, which allowed all parties to skip technical questions in the interview and instead focus on fit for both sides. The software engineering industry suffers from too great of an emphasis on absolute performance levels in my opinion. To pass a section of the CPA exam, one needs to score a minimum of 75. What do you call an accountant that passed every section of the CPA exam with 75s? A CPA.
Now even if you have a living and respectable proof that you have worked with this person and was good, it means almost nothing. Let’s five rounds of interviews. I have seen the same pattern even for internal transfers! Crazy!
As a hiring manager, when I get a reference from a good report I tell HR to CT the BS and trim down the process. I talk to the candidate, have tem meet a few folks from the team and we're good to go.
As a candidate, when I come in as a referred candidate and you throw your 2 hour take home in my face I'm walking away.
At this point, being referred just means that you have a good chance that your resume is at least passing the first review, but that's pretty much it.
I've been in the industry for ten years, but if I had to narrow down to the folks I'm not currently working with who worked closely with me, it's not that many people, unless we're going to people from 5+ years ago, and at that point, if I was on the receiving end of that, I don't know if I'd trust a reference that was that out-of-date.
Meta recently sent me their interview prep material and it says:
> In your tech screen, you'll be asked to solve one or two problems in under 35 minutes. Practice coding solutions to medium and hard problems in less than 15 minutes each to help you be ready for the constraints during the interview.
Less than 15 minutes?! The only way I can solve them in less than 15 minutes is if I've seen them before.
I think that's the point...
It’s hard for me to tell which companies require those style of interviews, but I’m not working in the Bay Area and have found when applying to positions remotely there, they seem to have a much greater emphasis on leetcode than companies in my area.
It's funny how they claim that a bad hire is devastating, and they can't rid of them easily, but somehow they can do mass layoffs and get rid of a bunch of engineers easily.
Yes, you are expected to hit-the-ground running on day one, but no one will immediately operate at their full potential. Even with all the shared best practices in the world, the secret sauce is the part you have to learn.
As an employer it's very hard to know if the reason for someone's uneven performance is due to ramp-up or if they are just not a good fit. Without a rigorous interview process, so many months would be wasted waiting to get a clear signal on that person.
That also doesn't account for complete cultural mismatches that cause instability in teams and hurt the impact of your other employees.
Another implied reason, good engineers want to surround themselves with other good engineers. So knowing its hard to get into a company signals to each applicant that the other employees there made it through that process.
Maybe that's because companies tend to hire whiteboard-master generalists rather than subject-matter specialists who may not be great at standardized technical interviews. ;-)
Also, if the company culture is ultra-bureaucratic, maybe the company should fix that instead of wasting months on every new hire.
Seriously, if a new engineer can't commit code within the first week, that's a company problem, not an engineer problem. Of course their code shouldn't go directly into production, but that's true of any new code. Give them something small to start, like some bugs to fix.
> That also doesn't account for complete cultural mismatches that cause instability in teams and hurt the impact of your other employees.
Technical interviews can't determine this.
> knowing its hard to get into a company signals to each applicant that the other employees there made it through that process.
I realize that's a signal, but it's not necessarily a good or accurate signal. I think it's mostly PR and hype. Reminds me a lot of fraternity hazing. Google engineers believe they're the best, and some of them may be, but some of them don't impress me at all. And as I mentioned, engineers tend to move from company to company anyway, so if Google engineers are "the best", they're constantly losing the best too.
I hope you realized that this should answer your own questions. Layoffs may be (relatively) easy, but firing someone for "you're just not cutting it" is much, much, much more difficult.
First off, most companies are loath to do large scale layoffs unless there are strong economic reasons to do so - many of the FAANGs have never had layoffs as big as the recent ones. So if your only chance to get rid of bad hires is every 5-10 years or so when there's an economic downturn, that's a problem.
But more importantly, while it's generally straightforward to fire someone who's flat out bad (as there is usually plenty of data to emphasize why they're bad), firing someone for cause who is just kinda mediocre is nearly impossible in the tech world in my experience. For example, if someone can do the job, but say is 50% slower than your average programmer (I've definitely seen this), it can be extremely difficult to gather enough evidence to fire that person. And it usually sucks for everyone involved, because often times these people who are slow are hard workers, but they're just not as capable as their peers.
One of the reasons you see the behaviors you see in technical interviews is precisely because hiring a kinda-OK-but-at-or-slightly-below-par is basically the worst kind of hire you can make.
It's actually not. When upper management is motivated to fire people, they get fired fast. Whether that's an individual person or a large group of people. We've seen this happen over and over. Self-imposed bureaucracy is the only thing that prevents fast firing.
> it can be extremely difficult to gather enough evidence to fire that person.
You don't need evidence. There's no such legal requirement. It's at-will employment.
And I don't want to hear about potential lawsuits. These are ghost stories, designed to scare, but ghosts don't exist. Show me the lawsuits. Incompetent people who are suddenly out of a job don't have the time or money to file frivolous lawsuits (which could get them blacklisted from the entire industry). The ratio of lawsuits to firings is close enough to zero to be negligible, and certainly big tech companies can afford to defend themselves.
I've worked with 200+ engineers and I know of exactly four that were fired for performance. But probably another 40 were quite bad and we would have been better off without them, they just didn't exactly meet the bar for 'so bad we have to fire them immediately'.
> the costs of decreased productivity and morale
I don't dispute any of that. I just mean that they can legally do it, and they don't have to justify it, they don't have to put employees on PIP, they don't have to give reasons why every employee was included. I mean, Elon Musk can basically walk into Twitter and haphazardly fire a ton of people. The consequences may be bad, but it's "easy" in the sense that he can just do it whenever he wants. Even more so for individual firings as opposed to mass layoffs.
I think this is the broken assumption—there are a lot of us who simply are willing to accept a sub-FAANG wage in exchange for a work environment/interview process that we feel respects us.
Could I use more money? Probably. But would I be happier with more? The research suggests I wouldn't.
I already make a 90th percentile income for my area, and I don't feel that investing additional mental and emotional resources in maximizing salary is the best route forward for pursuing happiness. I think that at this point there are other axes to optimize on that provide greater marginal gains to happiness.
I think for most people amounts up to something like 200-500k/yr (depending on COL) would provide increased happiness. Basically, if you ever have to worry about not having enough money, you could stand to make more.
Of course that doesn't mean it's necessarily worthwhile to work more to achieve that, that would be a personal decision you have to make yourself.
All the interviews at Google are 45 minutes, and when I interviewed only 2 had coding, so there was realistically 70-80 minutes of coding that day. I did maybe 15 minutes in a phone screen on an earlier date. Even if you did only 60 minutes for the whole process, you really aren't that far off from a typical FAANG.
I've had interviews with heavy LC and ones where I got plain old fizzbuzz and there wasn't much difference in staff competence or how quickly we delivered.
If anything, the place with the low bar had more well rounded peers I wanted to spend time with after work.
I will do take-home assignments (assuming 4~8h of work) or 4h+ onsite only if I am (quite-to-very) interested in the company. This is either the company is famous so I know them well, or there was a good interview process and they passed all of my questions/no red flags. If I'm on the verge of rejecting a company and they ask me for a sudden 4h+ process, sorry but not.
I live in Japan so it's been interesting as there's vastly different thinking companies, you have from the most modern flexible silicon-valley-like company (few, but there are) to very traditional ones that might even be confused when you reject them (again few, but some). Last time I interviewed I told a company I wasn't interested in their offer, only to receive an email later telling me they were not interested in hiring me. I could guess HR marking me as a no-hire was a lot better for that interviewer than marking me as rejecting them, but still made me laugh a bit of how much "no, I am breaking up with you" it sounded like.
Of course many carpenters work as contractors so that's why it seems a bit silly.
My buddy just got a job at a high-end cocktail bar as a bartender. Part of the interview process was asking him to mix a drink.
Gordon Ramsay has talked about how he'll interview chefs by asking them to make scrambled eggs.
Actors, even famous ones, generally have to 'read' for roles in order to land them.
Musicians interview for seats in symphonies by playing music.
MBAs have to do case studies to land jobs at high end consulting firms.
Hell I applied to Taco Bell as a kid and they made me take a short math test to prove I knew how to make change.
I could give similar examples for dozens of other jobs.
The cases where you don't need to demonstrate some skill in order to get the job generally fall into a few categories:
- There's some outside certifying body like the Bar, CPA, PE, various tradesmen unions, or all the licenses like a CDL.
- The jobs are undifferentiated so the workers are fungible (no special skills required).
- Job skill is immediately apparent (less than two weeks to know for certain if someone can do the job or not).
- The cost of a bad hire is low so you're willing to eat the cost and just cut the workers how don't work out.
Lately it seems take home projects are more and more common and these do take 8+ hours. If you want the job.
Here's what happens if you want to move to a "different place".
Say, you go to a different country: you have to spend upwards from a few month, but likely few years to confirm your degree. You won't need a stage, but you will not become an attending right away. In many cases there aren't even analogous positions if you move countries, unless medical systems are very similar, so, in most likelihood you will have to do at least a good chunk of residency training all over again. This will also be usually compounded by studying a new language to a very high degree as doctors are expected to produce a lot of written reports / engage in written communication, and, unlike programmers who almost universally use English regardless of the country they work in, doctors absolutely have to have good command of the local language(s).
Similarly, if you move between different medical organizations which manage hospitals. Sans the language requirements. However, within the same country hospitals will usually be more similar than between countries. Anyways, most hospitals will have fixed dates when job applications are processed, and even if you are extremely lucky and you don't need to redo any of your previous residency (both systems use the same PACS system, same or very similar internal organization etc.) you will still have to wait until the "draft" date. Typically, and due to competition, doctors will go to the hospital they intend to work at anyways before the "draft" date.
Even within the same hospital, if you want to move to a different department, you will still do residency, at least in part. I.e. say, you were already an attending in internal medicine, and you want to move to radiology: then maybe instead of 4 years, you'll do 3 years residency.
----
The above has a lot of compounding factors. Huge waiting times to get a position lead to doctors holding on to their positions with a lot more devotion than programmers. In many cases it's a job for life.
Because hospitals have to be in geographically diverse areas, they cannot, like programmers, all bunch together in one or two cities in a country and jump jobs w/o moving to a different apartment / house. A lot of hospitals thus include accommodation programs, which make it even harder to switch jobs.
It's very common for doctors to marry doctors. This makes some things easier, but it also means that if you need to switch jobs, then you have to do it in lockstep with your spouse.
Not in the least, if you move from a "less prestigious" country to a "more prestigious" country, you are almost automatically downgraded in your rank, and if you want the equivalent job, you'll have to jump through the same hoops the second time.
All of this is still not true in a most simple case, so getting job at a different hospital in the same specialization. Ex. in Poland most doctors are hired in multiple hospitals at the same time.
Genuinely curious how does this work? Do they get paid per the number of hospitals who hired them? How do they go to work? How do they know what hospital to go to?
PS. My wife is a doctor, and I had to live through what I described. So, none of that is invented, it's just what I see happen to her and to her colleagues. To make this more concrete, she was an attending in emergency department and wanted to switch to radiology. In her case this resulted in the full 4 years of study on top of about half a year of just showing up in the hospital and tagging along with the radiology team. (This was in Israel, one of the central hospitals). One of her colleagues was a transfer from internal medicine (also an attending), and he was doing 3 years of study to get into radiology. Another was a Russian emigrant doctor with about 10 years of practice from a hospital in St. Petersburg. He was also doing a 3 year of residency.
They also had two people drop out of the residency just during the year my wife was there (before she gave it up), and that's out of a group of six residents. One was a Brazilian emigrant, who eventually decided to go back to Brazil and another one was a guy who was an Israeli, but received his degree in Romania, which was cheaper, I guess. He just couldn't pull it up, and eventually was let go from the program.
The Russian guy was also on the verge of leaving due to some bad blood between him and the head of the department. The head was actively trying to sabotage him and make him leave for god knows what reason. The Russian guy though, despite having some sort of a chronic illness was spending multiple days in a row w/o leaving the hospital.
I mean, back to my original point: I saw nothing that could come close in the programming world. And the fuss people here make about home exercises is just a sign of being way, way overly privileged compared to the majority of the workforce. By which I don't mean to say programmers should suffer like everyone else, rather everyone else has to get better conditions. It's just of all people, presently, programmer should probably show more comradery with other paid workers instead of complaining about their own issues.
They have duty schedules, so they know where they should be at a given time. They have contracts signed with each hospital, like any employee. They can also work in private healthcare at the same time. It's just the case of setting up a schedules so they won't collide.
> she was an attending in emergency department and wanted to switch to radiology. In her case this resulted in the full 4 years of study on top of about half a year of just showing up in the hospital and tagging along with the radiology team.
That's normal because this is a specialization change. Not many doctors change specs or have more than one in most cases, at least in Poland.
How long is the lecture? 1hr? That doesn't sound bad compared to a 20hr programming assignment!!
Last 10 years no take home. Today it's coding something in a vc meeting, maybe in a web browser or coding env. Maybe beginners do some coding.
You signup for a new Amazon job. You are a senior developer you expect to make $400,000 with the stocks/salary. Your base outside of California is 139,000 or 129,000. After year 1 only 5% vests.. after year two 15%.. the average employment length is 1.5 years. So you end up with $140,000/150,000 for working 16 hour days. If you manage to stay 10 years you could retire..(you have to because at this point you hate life) but they don't want people staying at the same level so you need to get a promotion when the 4 year vest up or you will be at your base. Getting one takes the right project and is hard and requires a breakthrough project.
Most people 95% of developers never worked at a faang and those who have, on average worked for 1.5 years. Very few are still employed or seeking faang employment. Faangs make popular entry level position but very difficult to keep for life but if you can survive many years you usually leave the field or create your own startup because of burnout. Faang adjacent companies can be the worst of all worlds same issues worse pay/upside.
Well, I personally have always refused to do take-home interviews but happily will do live coding and systems design interviews.
For me it's about respect and power imbalance. A company asking me to do work without them putting in equal effort sets a tone for a culture I personally don't ever want to be a part of.
Like I find take-home interviews disrespectful.
Time-bounded interviews with an interviewer also there (aka FAANG style onsites with 4 hours of interviews) is far and away my preferred process, especially if I can do them all at once. One problem I've seen in a remote friendly world is companies wanting to spread the interviews out over multiple days.
You see people on HN balk at 500k+ engineering jobs even existing, so I think that's your answer.
Or startups. :)
In the world outside HN I very rarely encountered people who'd turn away from any kind of hiring process. Maybe one in fifty candidates?.. I don't have the numbers, but I think I only met such people twice in my life.
I bailed from interviews for different reasons, but I think that homework is a legit way to test someone's skills, so I wouldn't mind that.
The reasons I cut the hiring process short in my job hunts were most commonly:
1. Employer is an MS Windows shop. Sometimes it's hard to figure this out from the job posting.
2. Employer requires employees to use company-provided tools s.a. code editor, or antivirus etc. In other words, an over-reaching IT.
3. Crazy / not very smart / borderline criminal employer. Examples include a guy who had "scrum cards" deck on his desk and essentially showed me to the door when I asked if they used this stuff for real. Another one who couldn't get my homework to run, asked for a Docker image, couldn't run that either, asked for a VM image, couldn't run that either...
----
There's one litmus test I have when interviewing that turned out to be surprisingly precise, and I don't know why. I ask potential employer if they ever use git-merge. If the answer is "no", the company turns out to be intelligent people who are nice to work with, and if the answer is anything else, it turns out to be dysfunctional in more ways than just infra. They will have toxic culture, under-the-carpet skirmishes where each department undermines another department, while at the same time trying to do as little work as possible.
As you can imagine, unfortunately, I had to take jobs where the employer answered "yes" or "sometimes" etc. That's how I know :(
For all I know, I once joined a company where the policy was to only do squash-merges. I left from there at the brink of mental breakdown.
And what's wrong with merge? How do you even use git without merging?
In theory I don't mind them. In practice I've found that I more often encounter a scenario where I wish a change had been split to smaller commits rather than the scenario where there were too many commits to go through.
Anyway... I intended my example as an explicitly anecdotal evidence to counter the seemingly absurd suggestion of using Git without ever using merge. Feels like going back to subversion or CVS.
For more reliable code-bases you want to do the following:
* Run tests on each commit when accepting PRs.
* Be able to remove or edit intermediate commits, if you find that they've created problems afterwards.
* Only have one path from past to the future.
This is so because if want to use git-bisect, and instead of deleting faulty code you reverted it, the command will keep failing on the code that you've already fixed, and there's nothing you can do about it. git-bisect also has to follow one and only path from the past to the future because if you don't, then, at best, you get an combinatorial explosion of possible paths git-bisect may take, and at worst, some of these paths will fail, but others will not.
So, what ends up happening is this: people who use merges are like people who never clean up their apartment. For some it will take longer, than for others, but, inevitably, the apartment will become a filthy mess. But this is just a symptom of people being afraid of not understanding their code, being afraid of making big changes, undoing things committed to long ago.
This fear is usually an indication that people aren't good at the technology they are using. They would be too afraid to delete code because "what if it breaks something?" -- and nobody can tell authoritatively "no it doesn't". In a situation like this any change in technology s.a. using a different version of the same tool, or replacing the tool altogether will be almost impossible to implement because of the fear.
It's also usually very characteristic of places like this to be afraid of knowing / learning the underlying technology, the one that supports the entire company's stack. Eg. if it's a Python shop, then they'd be opposed to writing Python modules in C, even though this is how Python typically works, because they are afraid that they won't understand this code and one day will end up with a "magical" program that sometimes fails, but nobody knows why.
It's also usually the people who won't even try an unpopular technology, even if the benefits were huge, based on their fear of not having expertise to deal with it. Eg. XML schemas are hugely superior to JSON schemas, and if you want to validate your inputs, XML is just a better tool for this, but the company I'm describing will never consider using it because they are afraid of not being able to find people willing to work with DTD / XSL / RNG.
Such a company will never consider self-hosting, and will pay through the nose for the expertise of others, being mortally scared by a prospect of running their own infrastructure.
----
And... this is the majority profile. The problem is, this is not a winner's profile. It's a scrapping-by profile. It puts an individual programmer in the situation where there's no need and no reward for bettering themselves. Where management is antagonistic to programmers because they are in a conflicting situation, where on one hand they want to give customers more stuff, but on the other hand they are too afraid to make more stuff, since it may deprive them of the stuff they already have. So programmers are punished whether they do or whether they don't. It's where cargo cult flourishes. Basically, Dillbert comic before its author went into politics.
These developers may go their entire careers without ever reversing a binary tree on a whiteboard while juggling two bowling balls on a unicycle.
There wasn't growth in that team due to mediocre hiring and eventually all the good ones - left to other companies.
My current team is an infra platform and has lot of growth as IC. Everyone is learning something in-depth and are explorers - rather than blind sheep. The bar here is higher than the one for my previous team.
Our team requires you to know about whatever you talk on, not just usage but it's internals - why ? That's what we do daily. It can be about scheduler, checkpointing, auto scaling, concurrency, different data structures & algos, integrating with ecosystem, etc.
Even soft skills - like helping others, taking feedback, communicating clearly, etc.
Yeah so, mediocre will always be a burden to team.
If inverting a binary tree means swapping the left and right subtrees of every node, I wouldn't want to work with someone who can't do that either and Google is definitely right to reject him.
The question above, as clarified, is not complicated, nor does it rely on memorization or some "trick": anyone purporting to be a SWE should be able to write an essentially de novo solution to it.
(And in my own technical interviews, there are multiple questions, to specifically hedge against any one being "that one question a good candidate is going to miss because it's just not their day". It doesn't happen: it's either all or nothing.)
It is more likely that the interview process is broken and missing the right candidates, than it is that the interviewees are all mediocre. Most interviews are very non-inclusive the same way that the main track of school is becoming less and less inclusive. Different people need different methods to bring out the best in them.
Here's a thought experiment for you: if the interview process is so broken, why hasn't some tech company succeeded and become famous for an improved interview process, e.g. "Moneyball style"? My guess is because the process is not actually that broken, at least from the employer's perspective. I'm sure the interview process could be changed to be less regimented and more "inclusive", but that's also likely to reduce it's predictive power (i.e. you're more likely to make bad hires, and from a company's perspective that's almost always worse than missing out on a great hire).
Hiring is guessing. Firing is knowing. If the hiring process worked, we wouldn't have layoffs like we do.
Sure, an argument exists around "you shouldn't've hired that many people", but that is different from an argument of "the hiring process can't discern good hires". The former is a management & long-term planning issue, the latter is how interviews are conducted.
Sure, your day-to-day work may not involve manipulating binary trees. But presumably it does involve working with variables, objects, references, manipulating data of some kind... And if you're comfortable with the fundamentals of those, then this is something you should be able to figure out even if you've never heard of a "binary tree" before, once somebody has sketched it or shown you the definition of their TreeNode class, right?
It honestly baffles me how people consider this something which needs to be drilled or memorized.
There are absolutely algorithmic questions which would fall into that category. But if somebody considers this to be one of them - or something like "find the smallest number in an array" - then I have to question whether they have an understanding of the most fundamental concepts in programming...
Or if they get through each day solely using things they've memorized by rote, or looked up, and they don't really have any idea how any of the foundations they're building on actually operate.
How complex is homebrew ? Can no one else replicate it ? Why should a company hire for something you did that's simple ?
What are the skills he posses that no one else has ?
Learn your basica dsa stuff for gods sake people.
I went out of my way to avoid homebrew (still do) when I worked at google because it would reliably fail to complete some key operations in a dag, hence the interest in ensuring developers know how to do CS things.
Here's an example:
https://leetcode.com/problems/invert-binary-tree/
It's an 'easy' question. The solution is <10 lines.
Rejecting the guy because he cannot do a whiteboard brain teaser is like rejecting LeBron James because he did not make a shot at the arcade basketball game.
I'm not saying the guy would be perfect. Comparing him to LeBron James might not be a great example. Google might have other reasons to reject him.
What I'm trying to say is the current coding interview is a really poor mechanism to gauge a software engineer, especially when it comes to hiring one with real-world engineering experience.
People like to say this, but in my experience this is not true. It's just that people misunderstand the goal of technical interviews and they often are poor at evaluating their own skills.
First off, these giant tech companies have enormous economic incentives to improve their interview processes as much as possible. They also do a pretty rigorous assessment of the effectiveness of their interview process (Google, for example, has publicized some of their data). I'm not saying these tech companies interview processes are perfect, but I also have a problem believing they're so fundamentally flawed that these companies can't figure out how to fix them given the giant economic returns they get for optimizing their hiring processes.
Moreover, as some other comments mentioned, many companies (and individuals, myself included) believe it is much worse to hire someone who ends up not cutting it, than missing out on a potentially good hire. I can list out all the reasons why, but Joel Spoelsky has a pretty famous essay from a couple decades ago on the topic that explains it well [1].
Thus, it's not surprising hearing a lot people complain that they can do the job, but they aren't good at interviews. Because, from Google's/Microsoft's/etc. perspective, they're fine with a bit higher false negative rate if they can greatly reduce their false positive rate. And my experience matches that: I have never seen a candidate who did awesome in "whiteboard-style programming questions" who couldn't cut it programming-wise (they may have had other issues, but "coding productivity" wasn't one of them). Now, I certainly believe and have seen that there are some people who aren't good at these questions who can do a job well, but there are also a ton more people who can't do the job if they can't pass a technical screen, so hiring any of these folks means much more risk.
I also think that whiteboard-style coding questions help show a quality that is very important to businesses, even if those questions don't represent "real world" work. There are basically 2 types of people that do well at these questions: people who are just naturally smart and have a ton of experience to the point that they wouldn't even need to study to do well, and people who are of more "normal" intelligence/ability, but who can do well if they study a ton. Either of those two groups would likely do well in a programming role. So often I hear the complaint "I'm a busy person, I've got outside responsibilities, you can't expect me to spend all this time studying". And that may be true, but you'll be competing against people who are willing to study, so I don't think you can fault Google et al for favoring people who show a willingness to do more preparation.
1. https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... "And in the middle, you have a large number of “maybes” who seem like they might just be able to contribute something. The trick is telling the difference between the superstars and the maybes, because the secret is that you don’t want to hire any of the maybes. Ever."
But we have examples where the companies themselves have admitted that their past interview practices turned out not to work: https://business.time.com/2012/10/23/no-brainer-brainteaser-...
> They also do a pretty rigorous assessment of the effectiveness of their interview process
Companies do review their hiring processes, but actual experiments and data seem fairly rare. It's harder than you think. What experiment would you run? Hire a group of people entirely randomly, and compare their performance reviews after 2 years?
That's pretty much exactly my point. In the 90s, wide-scale hiring for software engineers was a relatively new thing - many companies were just figuring it out. And so they did some shit pretty early on that didn't make sense. But for all the times I hear folks pulling out the "Why are manhole covers round?" and "How many cars are there in Manhattan?" examples, I haven't heard these types of brainteaser questions being used for nearly 2 decades.
I'm not arguing that the FAANGs have some perfect, unassailable interview process that can never be improved, but I am arguing that so often I hear grumbling discontent from people who don't like the interview process, but rarely do I see much examination around why those particular hiring processes appear to work fairly well for the likes of Google, Apple, etc.
Yes, they did move away from it, but that doesn't mean we aren't now in the grip of equally bad fads.
> but rarely do I see much examination around why those particular hiring processes appear to work fairly well for the likes of Google, Apple, etc.
You assume they work well, but you don't have any data to support that. That's sort of assumption is basically where these hiring fads come from.
Now does being able to reverse a binary tree mean that you would fail at Google? I have no idea. But we don't really know if that was the reason he was rejected, it's just his own guess. There could have been other reasons.
Tree structure ? json, xml, protobuf, classes, functional programming, databases with foreign key, database internals, etc ?
Oh I forgot you also serialize and deserialize data - did you forget how that works ? Tree traversal again.
Do you know how organizationl hierarchy is structure ? It's a tree.
Do you know various maps and their usages ? We use them daily - it's very very important to know their internals. Hashing vs Trees vs Linked hash vs etc.
Google maps ? n-d trees ? Comparing data - merkel trees ? etc.
Every dev out there has common work with mine. But you won't be able to solve the problems that I face on a daily basis without thinking hard & without this dsa + concurrency knowledge.
Now, is it reasonable to ask these questions ? Heck yes.
I would rather hire an engineer with a strong business or user sense - reading between the lines of requests and anticipating future issues or uses adds so much more value in a real sense.
To me, these are great entry level questions because it is a good baseline for new grads when you have little work experience to judge. Past that, it is like making a lawyer take a mini bar exam for every new job - a waste of effort if you want to hire for specific skills and experience.
(I'm not sure Homebrew is all that well engineered, actually. Hard to tell, but I've had trouble with it and avoid it.)
I think what it comes down to is that nobody really knows what to interview for.
Generally it wouldn't really make sense to reverse a tree in practice (why not just build it the other way initially?) but it has a similar structure to other tree traversal things that could actually come up so it's reasonable to ask.
Companies that pay huge TC want to hire smart people not just an average joe. Sure, you can live satisfying career and that's your pov.
What about a company's pov ? Did you ever think about it ?
In my old team, I had to come up with a coupon distribution logic based on count, percentage, time, then generating reproducable random values that required to deep dive (algorithm) into library code & explicitly storing state in redis, then an application of dynamic programming in building as custom platform, atomic token validation, custom rate limiter algo, state machine, scheduler, distributed circuit breaker, etc.
In my current team, I had to read raft paper, zab paper, look into their implementation, make a poc with raft protocol, then autoscaling algorithms, scheduler algo's, different data structures, heck even the oss engine itself is DAG, heavy threads + concurrency stuff. Even now I come across new data structures and algorithms.
Clearly you don't know the entire industry, just because you haven't worked in such teams, doesn't mean these aren't important.
You are experienced in a bubble. The hiring bar for our team is higher than other teams and heck even for SDE3 - the requirement is higher. You would be very much surprised to know that even the senior members have research publications and deal with complex stuff.
Core teams like in AWS or GCP or Azure solve these sort of problems.
Who do you think will solve autoscaling (that's what I'm doing now) or managed scaling or network or security or any infra problems in these cloud platforms ?
As experience increases, we expect more knowledge & insights - doesn't mean to ignore basic coding stuff like arrays or linked lists or trees or graphs or simple message queues or etc.
If companies are paying competitive TC and there are multiple candidates, why not hire a smart person ? What's so special about doing regular normal stuff ? That's just a normal dev right ?
I can't think of any reason why anyone would ever do this. Just navigate the tree in the reverse of your normal direction instead.
Why not ask the much more interesting and potentially useful question of balancing a binary tree? Or do something else recursive, if that's what you're after.
Or even just when would you use a binary tree? Figuring out which data structure is appropriate for the problem at hand is the hard part, how to implement operations on the data structure is easy in comparison, you can just Google it.
That seems to be the wrong question though. It seems to be jeopardy style "question". You are not asking which data structure is appropriate for a problem.
Here is the answer, but what is the problem it solves.
Never in my live have I sat down and said: I don't know what problem is I need to solve, but I know the solution is a binary tree.
What's easier?
You usually aren't coding FizzBuzz in production code either
I would say you're constantly coding FizzBuzz. Looping, modulus arithmetic, and conditionals are all over the code I've written. At least with FizzBuzz you have a test of a person's ability to understand a task, break it down, and make sure the logic is consistent. With tree "inversion" it's not even a sensical request, it's utterly useless, and there are countless more interesting and practical ways to test understanding of recursion and trees. Knowledge that, I would bet, isn't even relevant for 90% of programmers, and if tree traversal were relevant then you'd probably want to jump to way more difficult questions (I'm thinking of the Facebook graph, for example).
I agree with the other commenter that it would make far more sense to ask questions like "you need to process data of this type, what provided data structure (e.g. C++'s STL) would you choose?"
Uhhhh... It eliminates competent, skilled people who don't have the time to memorize the latest cargo cult trends in hiring. Look in the article for a glaring example.
"Invert a binary tree" is actually considered an easy problem to test your basic knowledge of how trees work and tree traversal.
So would simply printing out a tree, and at least that's something a person might actually do.
Tree "inversion" doesn't even make any sense and at this point I'm convinced that the cargo cult is choosing it because it's the extent of their own understanding of trees and somehow sounds extra technical to them.
I agree completely it eliminates a huge swathe of people, mostly experienced and older people. FAANG employees, ime, are biased towards childless / single people with privileged backgrounds.
Tree inversion sounds weird when you hear it phrased like that, but in an interview it would be explained with an example (just swap the left and right children recursively).
It also seems to neatly split people into camps who think "this is trivial, and totally reasonable to expect somebody to answer, even if it's a little contrived", vs those who think "this is not practical, and you'd only know the answer if you'd already practiced it, so it's not fair to ask".
> It eliminates competent, skilled people who don't have the time to memorize the latest cargo cult trends in hiring
But given that you already acknowledged that it's pretty trivial, why would memorization be necessary for a competant person?
Every (recently with raft protocol) multi master distributed system out there interacts with Zookeeper for assigning leader and maintain configuration.
Do you know how the syntax or api calls look like ? They are node path in a tree. You want to store something ? That's a tree path again. You want to listen to some change ? It's tree path again.
As I said, most people are ignorant here and don't do "true" computer science engineering in daily life.
Most devs simply convert business logic into bunch of apis + adhoc implementation.
Did you ever work at a banking firm ? I've read their codebase - they are structured as trees, every damn thing is tree. It's a headache to navigate, code, heck even the objects are literal trees.
As I said, people are ignorant and think world revolves around them.
Honestly? Because that's harder.
The swapping question is basically a softball / FizzBuzz-style question to test the most basic familiarity with data structures, pointers/references, and recursion.
You the candidate, did not get the job because they did not like you ( e.g. you had a voice similar to the kid in school who was bully to your interviewer etc.). For most software positions out there, a relatively mediocre level of skills is sufficient. No fucking need to hair split on a person's technical skill.
If you truly care about the candidate first judge him on the 'cultural' fit. If he has crossed that barrier then the following advice from the article is a great one:
>Give candidates a heads-up about the attributes or topics the interview will cover and any other information you can reasonably share upfront.
All other advise in the article like paid assignment etc. are also great.
I would say it's more common to not get hired because you didn't have a strong yes. Maybe everyone thought you were ok but none of the interviews really wanted to argue for your case. There are also cases of people that are just obviously not qualified, like every single interviewer says 'no hire'.
You brought up (perhaps inadvertently) a good point here , most interviewers in the field are young, and their callowness is obvious, often you need senior people with the vetoing power to override their shallow analysis of a candidate.
If you have two almost identical candidates, but the other has better credentials, it’s difficult to turn them down - if you have some HR guidelines to follow
When you apply for a job as a doctor, they don't make you demonstrate surgical techniques. They take your board certification and ask you questions about what you've done in the past, things that went wrong and what you learned from that experience, and so on. They just assume you have the technical skills if you have the license.
Now, I'll admit that with doctors you are legally required to have the license, and we certainly don't need to go that far. If a small startup wants to take a chance on "unlicensed software engineers" or even offer to pay for the exam as a job perk (like a lot of law firms do for their interns), then great! But I can see a lot of time and effort saved if all the big enterprises would get together and come up with a national certification exam that you take once. Or even better, a series of exams for junior/senior/staff/principle, so neither candidates nor hiring managers have to waste time on tech assessments.
One of the keys would be making the exam inclusive for neurodivergent candidates, people with disabilities, etc. But this can be solved.
I'd be OK with this if all the answers were just shown to various companies so they could see strengths and weaknesses. If some company doesn't care about DP or low level OO design they can throw out those scores, etc.
I certainly got annoyed in my last job hunt where I had to answer some basic whiteboard easy level LC questions over and over and over. Thank god I got to skip the Google screen. If one more person made me do basic BFS or something I was going to freak out. I understand why they asked it, but I had to keep coding it over and over and over...
But it lets you skip the basic coding exams about data structures and OOP.
Let's be honest, you're not choosing 150K vs 400K based on how well they can reverse a binary tree.
We’re talking like 4 loc here…
But there are absolutely people with the title of senior engineer who cannot understand the concept of swapping some pointers and recursion.
And more importantly, like I said, you aren't choosing their salary and job level based on those skills. You're choosing it based on their experience.
Why would this hypothetical certification not have the same problems?
This would be different because they would have an incentive to be an accurate measure of ability so that companies would continue to require them. Presumably the big companies would constantly contribute to the curriculum, and the small companies would then benefit from that.
Medicine and the law are constantly changing, and somehow those licensing exams stay up to date. I assume the same methods could be used.
When I was working in the “real world” (I’m in consulting now), I could tell the people who just studied for a cert within the first 5 minutes of an interview.
I agree with you that most certs suck and are meaningless, but there are a few that are actually hard to get and meaningful.
Why would this not just transfer to the certification process?
I'm an actuary who had to take 10 of these for my profession (pass rates 20%-40% each sitting, takes 4 months to study for each one).
They come with the same complaints about false negatives, unrealistic/random questions, and fairness that technical interviews have.
That's when you already worked as a surgeon in a reputable hospital. The same way, the technical screens (leetcode) is generally minimal for hires that have demonstrated a clear pattern of success at reputable companies.
> We need an industry agreed-upon certification exam.
We already mostly do: proper CS/Engineering degrees have a pretty good signal to noise ratio for hiring.
Companies won't share their stats, but they know which schools and programs to target. Real engineering departments typically have job placement rates near 100%.
Anything more complex than that is probably job specific anyways.
Also, it should be noted that Amazon and Google don't interview for specific jobs -- they interview you for a job at the company and then you get placed. That would imply that at least at Google and Amazon, the skills are transferable across everything they do.
Lot of my thinking is based on visuals and emotions -- It's challenging for me to transcribe to English on demand and it interrupts my process -- it's somewhat like painting.
I always shine on take-homes since I'm allowed to be my authentic self. I'm enabled and have the full capacity to do my rituals, routines, and quirks.
Admittedly, this means I won't succeed in cooperative environments like pair programming. I'm better off left to my own devices.
Whatever happened to getting to know a candidates work? How about look into the work that a person has done and take the time to understand where a person's skills are. The problem with tech hiring is we have people trying to cut corners. So-called 'non-technical' recruiters doing interviews with a checklist, companies that treat people like hoop-jumping monkeys, and generally f*king idiots that won't do their job (they get paid for it, why again? They're not actually doing their job.)
Hiring is not a complex problem. The problem is literally incompetent people doing hiring.
Game changes if you actively contacted someone.. if you're no BS, assumption is you know who you contacted and why, hence only thing to do, once contact established is not to oversell and pay well.
Pay-well can constitue compensation as well as time.
It’s more conversational, and you don’t have to live in hypotheticals.
We all know that skilled engineers will learn whatever skills they need to on the job, so less and less am I interested in what they can do in the interview pressure cooker.
It's a bit orthogonal to the concerns in this article, but in some ways it's much more important.
What I wonder about is given an org that is able and willing to compensate at market clearing rates, how do they get the word out well enough to get engineers interested. Because the other big BS in hiring is the whole recruiting side of things.
I assume you mean empathetic. Same word is spelled “Emphethatic” later. (I tried finding a way to reach you privately, but your site “about” says you have contact methods on the left but, on mobile, there is no left…so here will have to do.)
It's so blatantly obvious that interviewers are basically trying to hire themselves, and will almost always select candidates whom they share the most personality traits with.
Also, I see the hiring process as similar to wanting to be a politician ie anyone who really wants to be one and is just really good at it should never be given the job.
The people who impress you the most almost surely have simply put a ton more effort into gaming the process with long leetcode sessions, live interview practices, and other bullshit tricks to convince you that they are the best person to hire.
With so much as stake, why wouldn't young devs spend tons of time working on interviewing skills and not really giving a damn about developing the real skills needed to be a goto resource at Big Software?
It's very much like taking steroids in professional sports...well no shit your taking PEDs when your career paths are either making generational wealth fucking with a ball or working selling Jordans at the local shoe store.
If I could give one minor persuasive writing critique to the author of this article though, I'd suggest not emphasizing inclusivity (which is basically bog standard, meaningless corporate drivel by now), but emphasizing that changing the typical interview pattern ensures that you're casting the widest possible net for talent. There's business people out there that couldn't give half an ounce of care for doing a solid for whatever the heck they might think a neurodivergent is, but if you emphatically frame this a bit different (solely as the company missing out on talent) I think the argument instantly becomes a lot more appealing to a pretty large group in the business world.
1) Doing a relatively shallow but wide survey of the technology I'll expect them to be responsible for. Because we use k8s, I steal the old "type google.com into your browser" question and make it, "I type 'kubectl get pods' and hit enter. What happens to make the list of pods show up on my screen?". From there, you can dive into basically any part of the stack you want.
I'll often ask them to explain to me what the difference between a container and a VM is, as though I were an intern, and then I'll ask probing questions about things they get wrong or things they leave out that I think are relevant.
This isn't to pass/fail them, necessarily (though some people have done so badly that they essentially failed themselves), but it's to see where their familiarity and comfort level is with the tech at hand.
2) talking through their resume with them, and doing a deep-dive into a couple aspects of their recent history - why did they do a thing? What alternatives did they explore? What was the reason they went with what they did? How did they implement it? What problems happened? Who did they collaborate with and what was the precise scope of their involvement? How did they measure success, and what was the follow-up?
I don't expect anyone coming in to have a deep knowledge of the tech stack we have. I do expect someone to have deep knowledge of the technology that they put on their resume, though.
My hit / miss rate is pretty decent. There have been a couple of times that I said no when I should have said yes, but I'm okay with that ratio.
1. If there are challenges, particularly if they are take home tests, it is important to make these reflect the sort of work someone will do without raising concerns that the work will be used by the company without pay. Candidates will spend time on relevant challenges and be happy. They will not be happy about irrelevant challenges. And interviews go both ways.
2. Dispense with "good questions" and go instead with "what do I want to know about a candidate.
3. Ask yourself before you start hiring, "What makes those who are successful at this company successful?" And from there, start building your interview structure.
Not every company will be the same, or will be good matches for the same candidates. The key should be to figure out what you need and use the interview to determine if the candidate actually is a good fit.
Unfortunately this cannot have data because it relies on a bunch of human judgment calls.
How many countless stories like that exist?
For example, people messaging for a chat "found your work on project X and saw your github account and found about your past projects, we are looking for some one like you", then on the day "oh sorry to let you know last minute but X and Y happened".
Bunch of time wasters! This is the reality!
Once in the job, it's funny to see who actually does the work. Zero contributions for days, etc.
There are a lot of people out there handling these processes and they are bad, really bad!
My own interviews have a list of topics I want to cover (non functional requirements, data experience, app design, infrastructure, etc), so I guess there is some structure. But I mostly run the interview based on their own experiences and projects they have worked on. So we will focus on applications and systems they have worked with in the past. And then I see how deep down those rabbit holes of their own system they can go.
They have different values. Different expectations of what is normal or important. "Culture", "team fit" and other bs.
Everything changes. New people come who don't know the past.
Why can’t anybody trust verifiable info these days - that you worked for $COMPANY doing some $JOB for $YEARS. Why are resumes completely thrown out the window and you start from ground zero on a whiteboard when you have over a decade of experience. I wasn’t practicing leet for the last decade, I was doing actual engineering.
I work in aerospace which often does a STAR behavioral interview (very light to zero technical interview) and I can honestly say that I work with some of the brightest people I’ve ever met.
So it's not that I don't trust that they didn't work there, but that doesn't mean that they can do a good job here. Every company has low performers, perhaps their previous employer was just bad at detecting them.
There's the rub. Women are disproportionately likely to spend time caring for children and have less free time available for hobbies. Personal projects might be a positive indicator that someone really cares about computing but the projects I've been praised for in interviews make up ~1/500th of the time I've spent developing my skills. It can objectively display the quality of code that a candidate can write but its absence isn't a guarantee that the skills don't exist.
HR and recruiting relationships are sparse at best.
It’s working well, and has avoided at least two false negatives since implementation within the last six months.
The purpose of the system is what it does. If desired state is not emerging, we must adjust and observe accordingly.
There is a large diversity in how developers are effective. When you force people into one funnel you lose the rest of the ecosystem. Meet people where they are, the only metric that should matter is effectiveness
If you still want "exceptional talent", but not algorithmic interviews, then you end up biasing towards white guys who have a ton of projects to show you.
Actually, I think this should be verifiable. Select some companies that we think have exceptionally high bar (you could use compensation as a proxy, acknowledging it's imperfect). Then classify them based on whether they do "leetcode" interviews or not, and check their diversity reports. My bet would be that the "leetcode" companies do significantly better.
* Caveat is that companies people think do "leetcode" actually usually ban questions that appear on leetcode.
Not really—if I don't have time to do an hour-long take-home assignment, what makes you think I have time to practice leetcode-style questions? The take-home assignment is usually testing skills that I actually use in my job on a regular basis, so I don't need extra preparation, I just need a block of time to sit down and do it.
I agree that expecting people to be able to show side projects is a mistake, though I'm not sure why you think that the bias there would be racial—I would imagine it would be much more a filter that excludes people with families and/or non-computer hobbies.
This seems off to me as well. Wealth, and schooling, are not relvant here. Just grab an old computer, install Linux for free, and off you go.
Loads of free tool stacks, github is free, etc.
There are of course a lot of caveats - experienced candidates should be treated differently from entry level. It’s also true that, e.g. Google has a weird fetish about dynamic programming questions and other problems that people are unlikely to figure out without having taken a class in them.
On the other hand, take home assignments take up time and room you might not have. It’s easier to do those when you’re a man in a developed country than, e.g. a single mother in Alabama.
So its basic, I think algo interviews are a good way to hire junior level engineers.
It's kind of a "choose your own adventure" style interview.
I explain upfront what the process will be, that during the interview "I don't know" is a preferred answer than BS.
Then I start with this question:
"Your task is to take some data in from a user, store it, and then present it back to the user. How do you do it?"
Based on their answer and follow up questions, I follow them down the path of their preferred stack.
This is rare, but I love it when prior to answering, a candidate asks me clarifing questions, like "how many concurrent users will the system need to support? What sort of performance is necessary?" Etc...
Many just assume that I'm asking about web development, so if they go down that path I challenge their assumptions, and follow up asking about their reasoning.
If they pick a framework, I ask what the advantages and disadvantages there are to that framework and how it compares to others.
Same with databases, etc..
It becomes pretty clear quickly what sort of level they're at. Many times the answer on a framework question is "that's what I used at my last job". That's not necessarily a poor answer, but it is informative.
I try hard to make it a casual conversation, like the sort you'd have with a technical stranger at a bar or something, though I understand that's impossible given the power dynamics and stakes involved. Still, I try.
So far, it seems to work pretty well for me. I have the luxury of keeping my team small, so I don't have to come up with a scalable "one size fits all" process, I'm able to keep it personal and relevant to the candidate and their experience.
So I'd add another criteria: interviewers need to be trained!
Plural: Criteria
SCNR
0. 45 minute homework/prescreen. Provide an (optional) pre-setup environment so it's mostly about coding and not about building/installing deps.
1. on-site where you chat about your solution, mostly an ice breaker/introduction to the team.
2. pair-programming to extend the homework or work on a simplified but real problem encountered day to day, open book
3. design review
4. code review
5. behavioral / case study
All of these can be pretty objective and don't rely on any memorization. All this should be pre-canned so individual proctors don't come up with their own questions and you're comparing candidates around the same prompts. It's amazing how few companies even manage these basic steps. I think most importantly the hiring should be done by a committee of actual practicing engineers - that means if you have checked in code in six months you aren't a vote on the committee.
1, 2 and 5 should be more than enough. You get to talk to them about their past expertise and even combine it with some design discussion, you get a pair programming session and a final casual discussion. Why do you need everything else?
I'd pick someone who really clicks with the team and is smart any day over someone who's brilliant but hard to work with.
Other things that are actually useful to consider (others have mentioned these too).
1) Your company culture is important for you to know; for you to codify and for you to communicate to your candidate.
2) In some companies, you will have tonnes of unsuitable applicants because of your brand, you should not optimise for people that aren't suitable - filter them asap
3) Your whole onboarding is a lot wider than just an interview. For many of us, we should be asking where/how we would expect to find our candidates and optimise those places. Do we visit hackathons? Is our recruitment page(s) clear and does it articulate what we want?
4) Your entire process needs to be like an interative development. Did you hire a bad person? What could you change to catch that in an interview? Wrong technical questions? Not enough about comms or culture?
5) In many cases, a company will hire someone that comes across well even if they don't necessarily tick the boxes so don't assume that you didn't get the job because you failed the whiteboard test, maybe you weren't as good as you thought?
6) Candidates need to think carefully about their approach to interviews, I would say more candidates than not seem almost entirely unprepared for normal interview questions and perhaps expect their ambience will get them the job! Study, swot up, never used owasp for web apps? Don't say that in an interview, spend 10 minutes learning about the top 10 and answer confidently.
7) Recruiters are in it for large commissions. They kind of care only in-as-much as getting a good reputation might help them but everything is second to money so don't expect them to place you well, don't expect them to sell you properly and if you get rejected, don't expect them to call you!
Yes they know its a 100% a trap, but good engineers won't be able to resist responding on the off chance it is real.
Don't click this... it is clearly a trick... you already knew they never have overstock sales. yet had to click this anyways... =)
"Process is broken" continues to be the theme where no parties agree whether a basic algo or leetcode or takehome is sufficient yet continuously reject professional designation.
Folks, keep in mind that at the end of the day, you are hiring a person, not an object with a bunch of methods.
This didn't make any sense to me, so I looked it up, and it seems exactly as senseless as I had thought. Is the 'inversion' not the same topology as the input?
my solution: "auto invert(regular_tree x){return static-cast<inverted_tree>(x);}".
Depending on these, you end up with different attitude and questions.
I quite like how things work currently. It's easy to tell the difference.
Being "good at the job" is actually bad because it reduces the team's required size.
Almost all the incentives revolve around this. Not only compensation, but entire hierarchies that revolve around EB1C dangling. (managers and executives at multinationals can get a green card faster)
You can't do that with a take-home (and I'm against take home as the signal to noise ratio is too low) because people will cheat and have them done by someone else.
I've heard horror story of a "senior" engineer from "his country's top school" being interviewed for a technical position by several non-technical managers and HR reps. They only included an engineer in the final round, which was basically supposed to be rubberstamped anyways. He was then asked to implement something trivial like fizzbuzz or wordcount on the whiteboard. The candidate then became extremely defensive and tried to argue that such task was "beneath him", arguing for a good 15 minutes why he shouldn't have to do it.
Then the dev just left the room and said that he used this question as a warmup with new hires and it typically takes them less than 10 minutes.
Now, a lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can't make any assumptions on what the candidates know. I'm not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast.
Now the hard part isn't so much to create an interview process that works well, but to create a pipeline that feeds into this interview process that has a high signal to noise ratio. In my experience, the best predictors of a good signal to noise ratio was to select for CS fundamentals, good references and offer above market comp. The latter is especially crucial now since there's no more "local market" to speak of now that remote work is a lot prevalent. The "local market's" best devs are working for SV firms at SV salaries mentoring SV employees.
[0] https://blog.codinghorror.com/why-cant-programmers-program/
[1] https://economictimes.indiatimes.com/tech/ites/95-engineers-...
When interviewing someone, I would still want someone to work through a problem live with me to see how they solve some problem. I don't care about the end result, I want to see how you're getting there.
- Steve Jobs
I think the real disconnect with the 'inclusive culture' boom comes because humans are involved so heavily in the process. The _idea_ is great, we want to be fully aware of our internal biases and avoid having them color our perception as much as possible so we do not shoot ourselves in the foot.
In practice, I have yet to see inclusivity programs at corporations be anything more than virtue signaling, and an opportunity to exclude others under the guise of "inclusivity" wink wink.
Remember 'affirmative action'? It's palpably Orwellian that inclusivity is newspeak; what we call it now.