Follow-up to “The dystopian world of software engineering interviews”
jarednelsen.dev
jarednelsen.dev
Fast forward a year and a friend of mine gets hired there. Turns out not only was my code pretty good, but had magically made its way into one of their production systems. The test cases that I wrote for my interview the year before word for word copied and pasted right into the prod test runs.
I know this because I used my name as the input for fname, lname trying to be a bit cheeky.
I have a theory. It's quite cynical, but it's taken me about 5 decades to arrive at it: life is a beauty pageant. I don't say this lightly. It was a very slow process to get to this point.
A friend, colleague recently interviewed at a very popular, pre-IPO company that is often discussed (positively) here. He doesn't look like your typical engineer, but he's literally the smartest person I've worked with. He gets shit done. He learns new things all the time. As his current manager, I told him I would give him a perfect recommendation. He had an "in" with a current employee. He aced 5/6 of the mini interviews, but "failed" one. Fine, it happens. But, something he said to me stuck in my head: the building of said company was filled with rows and rows of 20-30 white kids. He saw 2 people of color, and one was a greeter at the front door.
I don't believe the people that interviewed him were racist, but I think every person applies a "does he/she look the part" filter and if the answer is no, the mind finds reasons to not hire.
It's not just hiring that I think is pageant-like. So many things in life are. So much so, that I think people themselves actually live up or down to their physical features and characteristics. Look at the executives at a F500 company. Male and female. Why do they look so similar? I live in a somewhat affluent community. Why does everyone here look like they "belong"? This is not a gated community, so why does that happen?
Physical features are such a basic filter, applied to ourselves and others, that I don't think we are hardly aware of it.
EDIT: there are always exceptions. It's been my experience that exceptions are rare, though. And the exceptions sort of prove my point.
Can you elaborate on that?
I'd just download the next version of their app off the store and save a copy of it, then decompile it to prove my code is there and sue the living shit out of them
I guess that's not an option if you write server code for them though.. I'd probably negotiate to be paid for the work in that case
We are failing to bring certain softwares into existence by effectively requiring programmers to their effort into the World Wide Nether, finding the Kth smallest element in an unsorted BST for the millionth time, for seemingly no purpose except for it is "how things must unfortunately be done."
The part of recent grads getting job is more related to market conditions, back in 99 recruiter would be happy to hire anyone who have attended the programming 101 while in 2001-2002 no one even wanted to talk to 5 -8 yr experience candidates right after the market crash
Ultimately, though, you're right. The problem is that software engineering interviews aren't actually about software engineering. And, that's what really needs to change.
What's your reason for thinking there are more "software engineers" than "programmers"?
This is a funny topic to me. A decade ago, before Leetcode, Hackerrank, and Cracking The Coding Interview, it was common for developers to complain that software developers were losing touch with algorithmic knowledge. HN-type websites were full of discussions about clueless software engineers doing damage to companies by implementing O(n^2) algorithms that worked fine on small datasets but failed in production. It was popular to complain that modern libraries, frameworks, and programming languages were making developers too soft to write good code.
The industry responded by re-emphasizing the value of CS fundamentals and algorithmic knowledge. The training industry responded with easy tools to teach and learn these algorithms. The internet is full of easily accessible materials to teach these principles to anyone motivated enough to Google it.
Now, the popular narrative has flipped. Internet comment sections want to hate algorithms and suggest that programmers don't need to understand the basics. Just rely on the frameworks and libraries to do the right thing. Just Google the solution and weave the libraries together to do what you want.
> Whereas a young programmer in the past may have spent their extra days writing games in Basic or static web pages, hackers these days seem to do Leetcode until the wheels come off.
Young programmers don't wake up in the morning and choose between writing static web pages or grinding leetcode. There are more opportunities than ever before to learn whatever you want.
Have you actually worked with students lately? Leetcode style systems are a lot of fun for people. It's a straight to the point system that teaches algorithms and other fundamentals in self-contained brain teasers that are straight to the point. And it's trivially easy to Google for supporting training material. In my experience, the same people who are self-motivated enough to build web pages and games are frequently also interested in Leetcode style brain teasers.
People aren't choosing between one or the other. That's a false dichotomy. All of the highly driven students I've worked with have a range of experience from toy projects to Leetcode style learning challenges.
That’s not true for everyone, especially in such high stress situations as an interview. I like crosswords, but I wouldn’t want to do them in front of someone judging me for a job in 45 min while thinking aloud.
Heck I was too young when I started learning lisp (clojure). Took a long time to get used to non-functional constructs.
Things definitely changed once I got to university though. Competition for internships meant I had to get good at interviewing. The CS program I am in encourages the career focused so grinding leetcode is popular. One of my interviewers asked me a number, and used that to retrieve the n'th leetcode question from memory.
There does exist a small cadre of people, myself included, who find leetcode and similar interview grinds unethical. I'd rather spend my time having a life or learning/building interesting tech. Interviews are then less of a performance art and instead a genuine 1-1 to work through a problem.
I have a portfolio that consists of six figures LoC of code, across dozens of open-source repos; most of which I am the 100% sole author.
I've written a fairly large open-source system that is rapidly becoming the world standard for a specific (rather small) demographic. It deliberately uses old and rather primitive technology (which is a big reason for it becoming a standard).
I've written hundreds and hundreds (probably thousands) of pages of documentation and Web site entries, designed Web sites, and helped hundreds of people and organizations to develop online presence (all for free). I've run instructional courses on a number of different topics.
I speak Swift without an accent. I can write ship-ready applications for every Apple operating system. I've been writing shipping software for over thirty years.
I'm good with device control. I've written embedded operating systems (long ago -different beast from today's).
And, of course, I can PROVE it. It's all out there.
And no one bothers to look at my portfolio when I've been contacted to apply for engineering positions. I usually send a couple of links to repos and/or blog entries that apply directly to the position. My portfolio leaves absolutely no question at all about what I can and can't do, technically.
I once even had someone insinuate that I had somehow faked my portfolio. I'm not exactly sure how they thought I did it. Maybe I hired an Indian shop to write tens of thousands of lines of code, hundreds of blog entries, design multiple Web sites, and hack GitHub and BitBucket to fake a decade of commit history.
At least, that was the reason they gave for ignoring the links I sent, and instead, giving a binary tree "Draw Spunky" test (I don't do well on them; never have, never will. I spend exactly 0.0 hours on HackerRank, practicing).
I stopped trying to look for work. I don't need it, and I was getting tired of the grind.
I just set up my own gig, where I do the stuff I want to do. Maybe someone will want to work with me in the future; maybe not.
I don't really care, and I'm quite happy.
I am quite aware that most positions are for folks that don't have my level of expertise.
I just love designing and writing software; especially software that connects to devices. I'll do it; whether or not anyone pays me.
If they want to own it, though, and have more of a say in what it does, that's another story.
That would be believable if the same positions weren't being advertised for a year or more[1]; e.g. I've had one position[2] sent to me at least 5 times in the last year by different recruiters.
[1] In London, at least. [2] Interesting work and a job I'd want but unfortunately the corporate culture is 100% no-go for me.
That can't be true simultaneously with "companies are having trouble finding good enough candidates", which is often said these days.
What can be true: There are many high quality candidates, and many positions looking for high quality candidates, but they cannot find each other and overcome matching hurdles.
So you have candidates applying to a large number of positions, and positions seeing applications from a large number of candidates, just because the matching process is so inefficient. Then both sides can legitimately complain about the large amount of work and time it takes.
There are reasons why the current IT hiring status quo seems to be so paradoxical.
https://itnext.io/the-cloud-skills-shortage-and-the-unemploy...
Interesting link, thanks.
I've tried to reconcile that paradox by seeing it as a matching problem which requires both candidates and companies to be overwhelmed with attempts to find a good fit with each other.
However, reading the linked article gives another perspective on it. Which is, frankly, a worrying perspective for candidates, in that it suggests the labour market is shrinking and bifurcating (just like everything else these days) into "haves" and "have nots", where crossing from the latter to the former group may be getting more difficult with time.
I think maybe this is the unspoken issue with interviews - if a candidate was any good, they wouldn't need a job at all! So it's a given that all the people who apply are no good - the interviewer just needs to find the reason.
I hear this a lot from software engineers, but it might be worth thinking about the underlying expectation that anyone should spend time poking through repos on the internet, why they would, and how hard it might be to standardize a hiring process that compares you to other candidates. Contrast your own opinion of HackerRank with the hope that hiring managers, who perhaps often don’t have the time, interest, or even ability to read someone’s portfolio to determine the quality of the code.
As someone who’s done a lot of hiring, I often glance at an online portfolio briefly, but I find it to be a low-quality signal for whether I will want to hire the person, and even for the technical ability. If I don’t know you, I can’t tell from your online portfolio whether you’re the sole author and/or how much you’ve contributed to a project. I can’t easily tell if you get a lot of your code from StackOverflow, or write it yourself. I can’t tell what motivates you, whether you love programming or you are hoping for a high salary.
While high technical ability is a requirement for someone I hire, it’s not actually the primary requirement. What I need to know is how adaptable people are, how well they’ll work in a team, how quick they are on their feet, whether they’re friendly under pressure, a whole bunch of things that you can’t learn from a github account. Communication, attitude, curiosity, optimism, and cooperativeness are incredibly important traits, as is technical potential vs existing technical ability. I seek out ways to hire people who can (and want to) learn quickly, regardless of current skill level.
Personally, I’m totally convinced by your comment that your technical ability is high, where you say it is. You would easily pass my entry filter and make it to an interview. But after that, I’d want to talk to you, not sit at my computer looking through your portfolio.
I learn faster, and much, much more comprehensively, these days, than I ever did when I was younger (I suspect because of the vast context I got from experience). The main reason I'm doing my own thing, is because I love learning so much, and I see there's a whole world of stuff to learn.
StackOverflow is a critical tool for me, but only a reckless madman would stock a majority of their code from there. I use it to solve occasional vexing problems, from time to time, and always modify the (usually tiny) snippets I may glean from it.
All the rest is mine. I've been writing original code, often far ahead of its time, since the 1980s.
I would have been happy to work for a fraction of the salary a lot of folks far less capable would have asked; simply because I love doing this, and the challenge of the job would have been the most important coefficient.
But that's water under the bridge, these days. Once I accepted things the way they are, and just started doing what I wanted, on my own, I became much happier.
Would love to hear a little about how you're making ends meet and still doing what you want on your own. Are you selling software, doing contract work, making money some other way and doing software on the side...?
I also have two corporations, one with restrictions, but decent assets, and one with few assets, but a lot of flexibility.
How are performance reviews done where you work? It strikes me that you have to do all the same things to see if the portfolio of work an employee has done _for your company_ is up to snuff. Because i’ve seen a few places that just offload this to some numeric ranking based on the subjective (nay, political) perceptions of one’s peers through a filter of imperfect, variable understanding of how the numbers should even work, and I hated that process just as much as I hate most interview processes. In a way performance reviews are like interviewing to keep your job or get a promotion, so I’d find your approach to hiring to be a warning about how the culture operates there.
Everything about how a person works, in your subsequent paragraph, is true. But parent is talking about their currently existing body of work, vs every stupid take home challenge to do some menial transformation of a data structure wrapped in polished UI. Not as a stand in for their personality or culture fit.
Hahaha, well I can't stop you from making bad assumptions. When I say I believe that online portfolios are a low quality signal, I am speaking from experience. I have seen first-hand people that look amazing on paper who were bad hires, and also people who didn't invest in their online presence but were rock stars. I'd encourage you to think a little harder about what you really want out of hiring and performance reviews, and whether online material is actually what you want to be measured by. Not only is online material a low-quality indicator of someone's work performance, it's also much easier to game, in ways that are both subtle and not subtle. If you prefer that someone reads your repo checkins over talking to you, you're letting other people figure out how to make themselves look better than you with less effort. I'm not even talking about unscrupulous behavior -- when the metrics are online material, you will be competing with others' online material. People who write better will tend to "perform" better, people who produce more code will tend to look better (even though LOC is a terrible metric), and people who borrow and build on others' code will tend to look better than people who write their own.
Are you really certain these are the metrics by which you want to be judged in interviews and performance reviews?
The job I have now, thankfully, has the lowest overhead performance reviews I've had at any other job, because my team believes that annual performance reviews are not a very good system at all. We practice continuous performance reviews by working with each other and giving feedback on a regular basis. You should never be surprised by a performance review, or have to wade through the last year's work to see how someone did. If you're doing that, something is already wrong, it means you haven't been paying attention. When you have a system that reviews people annually, even if people are keeping records, it still tends to result in people changing their behavior for 1 month out of the year right before performance reviews. Doing amazing work for 10 months and then getting put on a crappy task can undo your whole year.
There is no online substitute for talking to people; it's much, much higher bandwidth than trying to read code checkins. Once someone has demonstrated they can code, I don't need to check for that again. What I need to know is why someone made the choices they did, what their philosophy about coding is, how they handle themselves during code reviews, how they work with others. I need to know what people are feeling, how they see themselves fitting in, and what their dreams and aspirations are. You can't get any of this from online portfolios, not when hiring, and not when reviewing employees.
I interview extremely well. I am empathetic, gregarious and genuinely like people. I have spent my entire adult life, getting along with some of the most difficult people in the world (long story, and has nothing to do with tech), and that has helped me to get along with people from a huge range of cultures and backgrounds.
Integrity is something that I take quite seriously. I live by a very well-defined personal code of ethics. It has served me well in my career.
It seems that I am not "cut out" for today's software development culture.
If you’re experiencing issues interviewing, there are lots of things that could be going wrong. It might be the types of jobs and/or companies you’re applying for. You might be shooting slightly too high or too low for your experience level. You mentioned plainly that you’d work for not much money, that could (perhaps surprisingly) work against you if you say it in interviews. It could even be sampling noise, just an unlucky string of interviews. But I’d hire someone like you on my team...
What's sad, is that people like me used to be represented by a certain kind of recruiter, more akin to a literary agent. They'd beat the bush, and get a portion of the take from that.
Apparently, this type of job no longer exists. I suspect that the big aggregator sites, like monster.com and Hired are responsible for that.
It's been a rather sobering experience. At one time, I had to fight off people trying to hire me away from my employer, to whom I gave a great deal of loyalty. When they finally rolled up my team, I completely understood it, and supported their decision; even though it cost my team and I our jobs. They treated us well, and with great respect. I had issues with the way they did things, in many cases, but in the aggregate, it was a great experience that I don't regret in the slightest.
I've been shocked at how standard recruiters haven't shown much enthusiasm. They are the ones that get in touch with me, gushing about my résumé, then quickly backtrack, for...reasons. I know why.
I've had to concede defeat, eat a huge platter of crow and egg-on-face, followed by a dessert of humble pie. I hadn't planned on early retirement, but there you have it.
Edited to add: One other thing. I tried sites like Hired and Upwork, and the contacts I got were 100% (not 95% -100%. Not one single legit contact). scams -some, rather scary. I have heard numerous people praise these sites, but they were all people looking to hire folks; not be hired.
To be fair, many of the scams weren't predatory. They assumed that I would be fronting as an American programmer, and shopping my work out to Indian programmers (who were the ones contacting me). I suspect that some of the contractors out there may not exactly be what they say.
> Apparently, this type of job no longer exists. I suspect that the big aggregator sites, like monster.com and Hired are responsible for that.
I’ve worked with Code Talent both when building out a team and when seeking a position myself and found that they work very much like this. I haven’t dealt with them recently, but they seem to still be going strong and doing what they do. I wouldn’t hesitate to reach out to them were I look to hire or find opportunities in the Denver area. Other than as a satisfied client on both sides of the table, I have no relationship with them.
But in terms of hiring, you as a newcomer will be jumping into a codebase and are expected to be able to work in it quickly. A symmetrical relationship for a hiring role might be to be able to find something to talk about in a candidate’s portfolio. Could even have the candidate point out the region of interest beforehand so you can have a good talk about it when the time comes. Not unlike a code review for a PR.
For new hires, I like to have an assigned mentor for the first year, it really helps ramp them up quicker, and have good material for performance reviews if necessary.
There’s nothing at all wrong with reviewing code and talking about it with the candidate/employee, that’s highly encouraged, and I do it constantly. I just can’t spend a lot of time looking at candidate’s portfolios without them being in the room, I don’t get enough information from it, and the information I do get might be misleading.
Then perhaps this person should not be interviewing in the first place? Why should an unqualified hiring manager somehow asses the candidates technical abilities ? This seems really bizarre to me. Imagine putting someone who is clueless about the medical field in charge of hiring surgeons based on some test score they took online, whilst completely ignoring their years of experience.
> I can’t tell from your online portfolio whether you’re the sole author and/or how much you’ve contributed to a project. I can’t easily tell if you get a lot of your code from StackOverflow, or write it yourself.
The OP has hundreds of citations in his portfolio. It's interesting that leetcode which can be gamed by a grad has more weight than multiple, well established projects. If you place this much trust in applicants, you will most likely be micromanaging them as well. This isn't the way to attract top talent.
> I can’t tell what motivates you, whether you love programming or you are hoping for a high salary. This is what the in person interview is for.
> While high technical ability is a requirement for someone I hire, it’s not actually the primary requirement. What I need to know is how adaptable people are, how well they’ll work in a team, how quick they are on their feet
All these are valid points, but the discussion here is regarding whiteboard algorithms vs actual experience.
I honestly think it boils down to a few things: - Companies are willing to waste the candidates time. - Copying FAANG style interviews serves as a marketing tactic for startups (we have the same hiring standards as Google)
And finally an unpopular opinion: - Asking Algorithm questions makes the interviewer feel smart, Especially if they are interviewing someone which is more experienced.
Look, this is the same as taking tests in school. Tests suck, but there's a reason we have to take tests and not just only turn in homework. The reason is that you need to demonstrate that you can do it on your own in a time-bound environment. Github doesn't demonstrate that even when you're amazing. Talking to someone who can ask questions about your abilities, and demonstrating your abilities on the spot is the way to do that, just like taking a test. I'm sorry it's less comfortable than posting online, but it's reality.
Leetcode doesn't have more weight than projects in my book, and I didn't mention it, neither did the top comment, so I'm not clear why you're saying that. Are you referring to the article?
> the discussion here is regarding whiteboard algorithms vs actual experience
The discussion in this sub-thread was about interviews & online portfolios. I wasn't talking about whiteboarding.
> Companies are willing to waste the candidates time [...] asking algorithm questions makes the interviewer feel smart.
I can't speak to all companies, you are probably right about many of them.
However, young grads today seem to feel entitled to easy interviews and job offers from all of them. What I hear in your post is some angst and frustration. I can understand that. I think it's a buyer's market. Getting a good job means figuring out how to stand out from the rest of the crowd, and it's really, really hard to do that with code alone.
It's true that companies are optimizing for their own needs, just as you are optimizing for your own needs. You can choose to see that negatively, as them wasting your time, or you can take more control of your interviews, and work to protect your own time.
The deal on offer is, if you pass the interviews, then you get paid well to code. Or, you can go start your own company and make your own money.
If you paid attention in school and understand algorithms, them you should think of algorithm questions as the easy part of the interview, something you can pass without problems. The questions are there to see if you care about algorithms and understand the basics. Don't take them personally, and understand that the questions are for all candidates, and they need to filter people out who don't know algorithms. So stop complaining about the easy questions, and start spending your time figuring out what the hard questions are and how to ace them. (Also, just FYI people who complain about certain kinds of interview questions is a small red flag in my book. If you love programming, algos might not be the exact thing you want to do, but it's part of the job pre-requisites and basic knowledge, so enjoy them instead of fighting them.)
... That's a software engineer's everyday job, specially when just starting a new gig. In fact, the "reasonable time" part is often a pipe dream.
Someone unwilling (or unable) to read and try to understand someone else's codebase has no place doing technical interviews. That's literally the first thing that a new hire has to do.
You might have half point if you were saying that you discuss someone's portfolio during the interview (which would be analogous with the new hire discussing the code with the rest of the team to better gauge the decisions made), but you've said that you dismiss it and instead defend tests.
Speaking of that defense of tests: surgeons also had to do tests in medical school, yet they're not asked to perform routine surgery as part of their interviews (that I know of). This is to say that you were using a false equivalency.
> I'm sorry it's less comfortable than posting online, but it's reality.
Evidently, it isn't. What makes it easy for you (which is ultimately the real reason for those pointless tests) is not the optimal process. It's, in fact, the entire problem discussed in the FA and the comments here.
> Leetcode doesn't have more weight than projects in my book, and I didn't mention it, neither did the top comment
From the top comment:
>> At least, that was the reason they gave for ignoring the links I sent, and instead, giving a binary tree "Draw Spunky" test (I don't do well on them; never have, never will. I spend exactly 0.0 hours on HackerRank [emphasis mine], practicing).
What exactly were you going for?
> The discussion in this sub-thread was about interviews & online portfolios
One of the main points of the top comment was the explicit and deliberate dismissal of the commenter's portfolio in favor of leetcode type tests. I refer to the paragraph I just quoted.
> and it's really, really hard to [stand out as a programmer] with code alone.
Interesting, most of the programmers I know of, I know of them because their code is great; I'm here mostly thinking of authors of widely used open source software.
To reference a famous anecdote: I'd hire the creator of homebrew on the spot. You wouldn't even give him an interview.
> you can take more control of your interviews, and work to protect your own time.
In other words: Potential candidates should make your job easier by continuing the drill this whole discussion is about instead of complaining about how ineffective it is. How much of a problem this whole charade is.
That's just borderline insulting, and it's pretty clear that it is deliberate. I'm surprised it flew here, specially considering the featured article(s) and most of the other comments.
> If you paid attention in school and understand algorithms, them [sic] you should think of algorithm questions as the easy part of the interview, something you can pass without problems.
You called yourself a great programmer, I'll make no comment on that, but you've given me a good idea for how to screen which engineers should participate in interviews for when I have my own company: I'll ask them what they think of algorithm questions and I'll ask them whether they think that what is taught in college should be enough to pass those questions.
If they respond on the affirmative, I'll ask them the most obscure thing I can think of from any of my physics or calculus classes. Hey, after all, they "studied that in college", they ought to remember it even though it would be a gigantic coincidence if they ever actually had to use that knowledge in their job, and the question should be an easy point, correct?
Hopefully that will be a teachable moment, on how responding on the affirmative was wrong, maybe even unethical (because I sure wouldn't have asked that during their own interviews, with one caveat).
And this is, of course, ignoring the whole point of the featured article and the original one this is a follow up to: algorithm questions have devolved into being far more difficult than anything anyone had to do in college and they're something that 99% of candidates would never have done, or needed, in any previous job.
> The questions are there to see if you care about algorithms and understand the basics.
They aren't, GP comment was on the money by saying that they're just there for the interviewer to feel smart.
> Don't take them personally, and understand that the questions are for all candidates, and they need to filter people out who don't know algorithms.
Filtering out people who don't remember (as opposed to not knowing; for remembrance only a short refresher is needed, but that's something that a coding test doesn't give the time for) algorithms is a terrible practice unless the company doing the hiring is explicitly researching for algorithm design and creating knowledge as opposed to applying existing knowledge.
Or, in short, most people won't ever need to dive deeply into algorithm design in this field (most, if not all, of the algorithms needed are already baked into the commonly used libraries), meaning that filtering out the ones that don't remember is pointless at best, self-defeating at worst.
For the very rare cases (outside of the aforementioned companies that need to create new stuff for their products) where reimplementing an algorithm is needed (and there better have been enough research to know that it's not an example of NIH), a couple of hours of getting reacquainted with them is often enough.
> So stop complaining about the easy questions
You've given no reason to. In fact, by rationalizing it, you helped me see with even more clarity how much of a problem this is.
> Also, just FYI people who complain about certain kinds of interview questions is a small red flag in my book.
That's good, every person you rejected for that reason (along with the ones rejected over tests, specially if they had portfolios) clearly wasn't a culture fit and they are better off in a different company.
I just wish they could have dodged the whole hiring process you employ. Then again, I wish that for everyone, myself included.
> If you love programming, algos might not be the exact thing you want to do, but it's part of the job pre-requisites and basic knowledge
They aren't prerequisite for 99% of the jobs and I would say that, if you love programming, you would, well, program and let everyone know you do, like by contributing to open source software and building a portfolio. And yet you would barely spend a second of your time looking at someone's portfolio.
So whose criteria for "someone who loves programming" is the right one?
Finally, a disclaimer: I often do well in those stupid tests and have never spent more than a few weeks in between jobs, while my online portfolio is pathetically barren. Just to put in perspective that one can hate everything about the problem and rightfully complain about it while still having to play, and mostly knowing how to play, that awful game.
I can see that you’re frustrated by interviews. Why don’t you tell me more about your experience and what about interviews isn’t working for you? You said you’ve never gone long in between jobs, so why are interviews bothering you so much? If you’re just commenting on the awful game, why are you taking it personally and insulting me?
You took much offense to my explanation of algorithm questions without acknowledging that I agreed with the GP comment that many companies might be doing it for the wrong reasons. Still, that doesn’t make it a good idea to just speculate on the reasons companies use algorithms questions. Just because you feel like they’re pointless and interviewers are trying to feel smart doesn’t mean it’s true. Jumping to a negative conclusion isn’t the best idea, even if you’re right.
The person who’s criteria matters for loving programming is you.
You're the one defending rote tests (that literally only test for technical knowledge), not me nor anyone who replied to you (as far as I read).
From my previous comment:
>> You might have half point if you were saying that you discuss someone's portfolio during the interview (which would be analogous with the new hire discussing the code with the rest of the team to better gauge the decisions made), but you've said that you dismiss it and instead defend tests.
I value discussion, that's ultimately one of the things that matters most for a developer. And, mind you, thanks to git platforms (GitLab, GitHub, etc.) this is also becoming more transparent in portfolios, like by looking at how the candidate handles merge/pull requests and issues.
> your many personal insults directed at me.
Could you quote which sentences in my previous comment were personal insults? I'd like to know where I phrased myself poorly and apologize and/or explain my intent.
> If someone who was interviewing you read your comment, do you think they’d want to hire you more?
If someone who was interviewing me disagreed with any of my points, I guess not. Then again, I wouldn't want to work in such a company either; making it a win win situation.
I fail to see how this sentence is apropos anyway, HN isn't a job board.
> You said you’ve never gone long in between jobs, so why are interviews bothering you so much?
That's all but spelled out in the article and most of the comments, but I'll share my experience explicitly: I am yet to have a job, that used those leetcode type tests or generic coding challenges as part of the hiring process, where I needed to use the knowledge required for the tests or the challenges in the actual job.
It's a gigantic waste of time, testing for all the wrong things.
> Just because you feel like they’re pointless and interviewers are trying to feel smart doesn’t mean it’s true.
I'm yet to find a counterexample in any job I've had. You seem to be mistaking your own experience (assuming you actually do those tests for the "right" reasons) as the norm. Though I guess all of us are a little bit at fault of that.
> Jumping to a negative conclusion isn’t the best idea, even if you’re right.
You are the one who told someone else to stop complaining and even made another not-apropos comment about how complaining is a red flag. There's no way to interpret that other than negatively. If there is, I would love to read what you meant by that.
> The person who’s criteria matters for loving programming is you.
Alas, evidently it doesn't, because even hiring managers/interviewers/recruiters who comment in HN posts about the awful state in hiring for software engineers think that rote algorithm tests are of utmost importance, and I don't particularly agree.
If I have to become the culture in order to participate in it, then I will just take my toys and play in my own sandbox. I won't compromise my personal ethos, just because everyone else is.
As a manager, I took my Responsibilities very seriously. I worked for an extremely frugal and conservative corporation, and had to fight like a badger to get headcount. Making sure that the person that filled that headcount was top-notch was critically important, and I spent a great deal of time evaluating résumés and interviewing candidates. I would have killed for information like what you can get from a well-stocked SO story. Each candidate was a potential "family member." I kept them for decades. Japan wouldn't even recognize their existence until they'd been there for over a year.
Have you ever seen a designer get ready for an interview? I was trained as an artist (way back when), and know the drill.
They bring a large (usually black) case with them. It contains samples of their work; often including things like sketches and layout exercises. They go over the contents of this case with the interviewer; often telling stories about how they addressed some kind of issue, or the creative process that went into the work. I guess they may bring a laptop with them, these days.
No design manager in their right mind would ever think of ignoring the portfolio, and, instead, throw a matchbook on the table, and ask the designer to "Draw Spunky."
I'm not saying ignore people's portfolio. I'm only saying I don't spend a lot of time doing it before I talk to them. You also understand as well as I do that code portfolios and art portfolios are very different things, we can't compare those two types of interviews directly. It's easy to get a sense of quality from an art portfolio very, very quickly. At the same time, it can take days to understand whether a code portfolio has high quality.
You, as someone who'd done a lot of hiring, I think understands that a hiring manager doesn't have 4 hours to research each and every candidate at the resume phase before interviewing them on the phone, while also trying to write code and manage other coders at the same time. We take the best candidates we get, rank the resumes, look over the portfolios, reject any candidates that aren't prepared for the job, then call the top candidates and chat on the phone for a few hours, after that bring in the top from that set for interviews, which can be multiple days.
I also take hiring extremely seriously, and I've never had to fire someone, because we spend a lot of time talking with the candidate and making certain they'll fit in well on the team. That time spent talking is spent talking about their portfolio, having the candidate explain in their own words why they made the design decisions they made, why they chose to work on certain projects.
I would be willing to bet that in a face to face conversation, we would be agreeing damn near 100% on hiring philosophy and process. The problem here is it's hard to get across my motivation online in text. :P
My writing tends to be a lot more casual, and even a bit humorous. I like humans, and believe that the human relationships we make are the most important artifacts of our lives.
I think the root cause of all interview issues is that interviewers aren't incentivized to care. You're never going to get promoted or even a good performance review for reading a candidate's code or ask them meaningful questions. On the other hand, it takes little effort to pick a couple algo questions and use them for everybody, and they still provide some meaningful signal. With alternative interview methods, apathetic interviewers will just rely on their intuition, which is how a "good ole boy's club" is formed.
A lot of the reason that I stopped looking for work, was because, almost instantly upon finding out that I am afflicted with eld, the microaggressions and veiled insults begin.
It's kind of amazing. People contact me, telling me how awesome my résumé is, then quickly backtrack, as soon as they find out it comes with grey hair. I guess the industry is full of 30-year-olds with 35 years' worth of experience.
It was really getting me steamed.
I'm not rich, and could always use the income from more work, but not badly enough to put up with that stuff.
I just make sure that people know that I'm "ancient," right up front, so we don't waste each others' time.
Other non FAANG/Unicorns that don’t pay as well as FAANG can drop the practice of asking DS&A questions. But for those companies that pay really really well, there will be insane amount of people that are willing to get a job there, and DS&A is a “May the strongest win” filter (although luck play a very strong role as well, yes there are stories where a junior job applicant get 4 Hard Leetcode out of 5 questions). I happily oblige. I happily will get through 1000 Leetcode and roll the dice (apply multiple times) if I need to in order to get that salary. No I’m not willing to get lower salary than FAANG/Unicorns, that’s why I’m doing this.
For companies that don’t pay as well as FAANG, just drop your DS&A interview practice, or just lower the bar, otherwise it is a waste of time for either the companies or the applicants.
EDIT:
TL/DR. The whole answer to this DS&A debacle is only 1 thing: SALARY. End. Period. No other factor matters.
You lower the salary, no one will bother with DS&A. You increase the salary, every single person on earth and their dog will study DS&A 24/7 to get that FAANG salary. Don't believe me? Head to Leetcode forum and reddit cscareerquestions. Actually we don't even need to go that far, just look at this topic on HN every single time. It is all about SALARY. Full stop.
It is as follows:
Offer ridiculously low pay, such that only basic expenses could be paid for, like basic housing...
Then, see who shows up.
Whoever shows up... really wants a job.
You don't hire them yet.
You implement your own corporate bootcamp, where you give then coding assignments. These start easy, but progress, over a number of weeks, to those that are similar in form and difficulty to the type of work that the corporation does...
A supervisor or group of supervisors reviews their work after every week.
People who do not meet expectations are let go.
What you're left with after that process is your new hires.
That's how you cut down on frivolous resumes.
That's how you weed out the casual job seekers from the purists.
The purists don't care about money as much as working with the right people.
It's not for everybody.
People who need salaries obviously, beyond the most basic of living expenses, would be unable to apply...
If you want to be a dick and systematically screw people, just hire an army of agency contractors in some out of the way place.
...The intended goal is to figure out who is intrinsically motivated (i.e., curiousity, innate interest, desire to learn, etc.), from those that are extrinsically motivated (i.e., salary, money, title, career, corporate position, etc.)
I’m also not an idiot and I can readily bubble sort a low offer against my current comp and against the other offers.
Hypothetical scenario, tell the startups/VCs to pay more for software engineers, just a little bit lower than FAANG, then two things will happen: 1. Either FAANG will even increase their salary even more to attract type A types 2. A huge number of applicants of varying quality will flock to CS as an industry lowering the overall quality, forcing companies to do hard DS&A questions to filter applicants.
I posted what I said as a note to my future self, if/when I run a software engineering company... if/when that happens, then that's how I'm going to handle hiring...
This will differentiate companies looking for true partnership with an engineer vs those who need hamsters in a wheel.
2) Pay them to spend several weeks taking tests.
3) ?
4) Profit
I'm curious what step 3 is?
Also you say that "The purists don't care about money as much as working with the right people." but you're intentionally selecting for the wrong people, so is this intended to avoid purists?
Then why do they also apply this to candidates they reach out to? I've never applied to FAANG, only interviewed after (in-house) recruiters or internal referrals. It always seems to be the same bullshit.
Jokes aside, my point is, you can only get better, and when you get better, a non FAANG interview is actually a piece of cake, and you can just job hop every year until you maxed out your salary band, rinse and repeat.
Repeat after me: It can only get better, and one day you will actually conquer FAANG.
Take this with a grain of salt, because my experience may differ from many here (never worked at FAANG, I spent a lot of time at smaller startups in Boulder and am currently in Miami) but I have only been put in front of a whiteboard one time during many dozens of interviews and it was a generic logic puzzle to see how I performed under pressure.
Having been on both sides of the hiring table now many times, there is only one approach that makes sense to me:
1. Speak to them over the phone to make sure they are decent human being, verify again in person.
2. Be upfront with general compensation ability and expectations so that neither of you waste the others time.
3. Do a pair-programming exercise where they build a small piece of software that mirrors something that would be working on as a day-to-day there. Make it clear that finishing is not important, you just want to see their thought process and how they communicate. Ask them to refactor parts of it at some point to see how coach-able they are and how they respond to criticism.
This strategy has worked exceedingly well for me, and when I was on the other end of the hiring table the companies whose process generally looked like this were great places to work with good people.
Algorithm questions are a hazing ritual, and so many junior devs get sucked into wasting hundreds of hours practicing them when they could be building real-world skills. This practice has got to stop, IMO.
In my experience, the gap between the HN comments section and the real world is massive when it comes to hiring practices.
In the world of HN comments, most people tend to identify with the candidate rather the interviewers. The interview candidate is the hero, the underdog, and the person we're supposed to root for. The interviewers are bumbling fools, drunk on power and dead set on abusing their power to find flimsy excuses to reject candidates.
In the real world, most of the senior engineers I know have spent considerable time on both sides of the interviewing table. At different points in their career, they've been the ones asking questions as well as the ones being interviewed. This leads to a lot of cross-pollination of interview techniques as well as different perspectives on what works vs. what doesn't.
We can all agree that interviewing and hiring practices aren't perfect, but it's much harder to suggest better alternatives. At one point, I tried directly asking candidates how they'd prefer to be interviewed in a way that best highlighted their talents. A surprising number of people defaulted back to standard interview practices, but I also had a number of "just trust me when I say that you should give me this job" type answers.
These online discussions inevitably skew one way, because no one wants to be the bad guy defending the current imperfect status quo. Even the author of this article artfully dodges any request to suggest alternatives, instead falling back on the safe and secure "we have a lot of work to do" response.
This is really cool to hear from someone else. I've done a similar thing, asking "What do you think you could do during a technical interview to best showcase your skills?"
I too got a surprising number of terrible/generic answers, though it very likely could have been nerves in the moment.
I'm one of those engineers. The company I currently work at has no problem finding and hiring competent engineers. I also have no problem suggesting something better than what most companies do. And yet, whenever I interview at other companies[0], the most common interview experience I still have are the horrible ones described time and time again here.
The sort of "cross-pollination" you refer to is not widespread.
[0]: A few times a year, just to keep my feet wet.
Is that really so far out? In many lines of work, you're expected to be trained into the position. My experience in software is that each company thinks they want someone who already knows X, Y, and Z, and can hit the ground running with no training.
But in reality, the hire will likely spend more time maintaining P & Q, and working on S, and will need quite a bit of domain knowledge as well as knowledge of the company's custom code and practices and internal organization before they're fully ramped up. And by the time they'd even get to work on X, Y, and Z, the company has likely moved on to A, B, and C, which almost everyone there is just learning.
It's a job that requires constant learning, and you're going to need to train your new hires whether you like it or not.
That's why when I'm called on to be the one asking questions, I mostly look for whether they enjoy learning and experimenting, whether they can have fun with old stuff and convey the differences between that and the new stuff (and why they're different), and whether they're open-minded or stubbornly convinced that they know the one true way.
That's the sort of thing you need when your old system has a decade+ of custom code and the dust hasn't even settled on the new system enough for the pros to know the right ways to do things with it yet. What you really need is someone with basic skills that's also open to being trained and learning new things.
Not just the algorithmic part, even the part leading up to it.
You have to impress a recruiter or an HR drone by listing all the buzzwords that are currently trendy.
Then, even after the algorithmic questions and interviews where you prove you can DO the job, you get rejected.
Every company I've worked for had trouble hiring.
I'm working for a company that also has trouble hiring right now. Last week, the person interviewing our latest candidate told me that despite the candidate doing perfectly fine, he couldn't come up with something that he felt excited about doing in his current job, which was a red flag for him.
I tried explaining that perhaps that's something you shouldn't expect from someone who's at that point switching jobs. We're still discussing how hard it is to find good people.
Most of the companies I've interviewed for also have trouble hiring, yet I've been rejected for the most asinine reasons. These companies have the same positions open years later and not due to growth.
All this rant is to say, we're mostly doing this to ourselves.
I don't have a better way of doing things, but I can't just blame a handful of big corps for this, as if their technical interviews weren't designed by technical folks themselves.
What do your counterparts at the company think the source of their hiring difficulties is? If not flawed process?
But when you reach the final stage with 3 people who you know can pretty much do the job, who you know have successful careers working in successful companies, and you drop them because they didn't exactly behave the way you would prefer them to in a high stress scenario, and keep looking, I think you effectively forfeit the right to claim that's your problem with hiring.
I didn't think of the solution you had in mind, because I've identified another problem that you didn't consider. And somehow that takes me out of the running. Good luck with that.
Not to mention the nebulous and vaguely discriminatory "cultural fit" problem. So, even if I manage to ace the technical, I still fail.
Needless to say that company had a lot of problems with technology. But they seemed to be good at making money strangely enough.
This is my least favorite part about discussing interview techniques.
It's taboo to say anything other than "interviews are broken", as if there is some alternative perfect practice that companies are voluntarily choosing not to use.
If no one, including the author of the linked article, can think of anything better, doesn't that mean that companies are already using optimal practices? At least from our current knowledge of interviewing practices. I always have my eyes open for new research, new suggestions, and new trends, but it's getting old to hear the constant chorus of "interviewing is broken" with zero attempt to suggest improvements.
That doesn't mean current practices are perfect or that we should stop trying to improve them, of course. It does feel a bit like politicians going on record saying that they're "deeply troubled" by something without proposing alternatives, though. It's the safe thing to say.
Not necessarily, it might just mean that the more optimal approaches are not obvious. It seems extremely unlikely that the current approach is optimal.
>At least from our current knowledge of interviewing practices
You have three promising candidates? Instead of trying to extract signal from gimmick interview tactics, hire them all and put them on a 3-month contract with the opportunity to renew to full time. Then you're giving them real weork instead of easily crammed questions, and remove some of the anxiety affecting interview performance.
There’ll be no shortages of applicants for quite some time but really destroys the society. So no.
No unless I’m guaranteed that upper echelon slot to judge them — no I wouldn’t want to live in a society like that.
On the other hand, I think it may be a good idea for the company to consider:
- either hire fulltime somebody, who has solid track record and "prooved" themself during on interview
OR
- offer 3 months contract to someone, who is not that good at passing interviews, so they can prove themself doing real work.
And the funny thing is, that got us a pretty good set of people. Even funnier: both the very best of them, and the absolute worst of them, eventually ended up at Google.
I guess if you can spend what Google spends; have a ruthless hiring process; can afford to carry dead weight indefinitely; have a standardized engineering culture; and have an endless supply of projects covering every conceivable skill level and impact to the company -- then you can probably hire your way to whatever FAANG-like cohort you think will work.
But if you have those things you're probably already a FAANG-like company.
For everybody else, I am not convinced there is a better system than "everybody use your best judgement, and have people who give a shit do the hiring."
I'm part of another online community that has a strong emphasize on social justice issues. These "just wing it" hiring practices are one of their biggest complaints about tech companies. There's a perception that the less well-defined and structured your interview process, the more likely it is to produce vaguely biased results.
Whether that's true or not, it would be a tough sell to implement a "just use your gut" style hiring process at a major company in 2020. There is a strong push for standardized and repeatable interviewing processes, complete with sufficient documentation to support hiring decisions.
I personally find some of the current (if I can call them) standards of tech hiring to be very narrow in which skills and capabilities they test. It presumes a lot—particularly that one has time and resources to commit to spending time studying enough of the spectrum of CS subjects in order to pass a given question from the rather large field. (By this I’m referring to the classic “white board a <somekindof>sort algorithm purely from memory” process that has become the de-facto standard)
What would you propose as an alternative?
I’ve been involved in training new engineering managers how to hire and interview. Everyone starts with the best intentions, but reality quickly forces some compromises.
The bottom line is that you only have a number of hours in which to evaluate the candidate. The longer and more comprehensive you make your interview processes, the stronger signal you get. However, the longer your interview process, the more you’ll find people dropping out of the hiring pipeline due to lack of time or willingness.
I’ve worked with companies that tried to implement week-long paid interviews where candidates pair with employees to solve real problems. It’s great! But only when it works for the candidate. You can’t realistically expect busy professionals to take an entire week off of their current job to spend time with a company that may or may not hire them. Monetary compensation only goes so far when you’re forcing someone to choose between spending a week of their vacation interviewing with your company or spending a week taking their family on vacation.
I think a general chat about a broad problem and the interviewee's thought process of how they would approach it can tell you much more than whiteboarding some memorized algorithm.
I mentioned to a colleague that what I value most is someone who is creative. There are many candidates out there who know all the buzzwords or can discuss features of a programming language purely from memory, etc.
This can make for some impressive interviewing, but it tells nothing of their ability to use that knowledge to solve real problems, or their creativity in how they use their skill and expertise when faced with new dilemmas.
Sure, you could contest that the creative side isn’t necessary for certain levels of engineering, or that only tech leads require this as a mandatory trait, but I would argue that it is far more important than what most interviews test - rote memorization.
I've always tried to do that... until recently.
With a new recruit, this is the fastest way to nightmare discussions. He always emphasizes the cons of my proposals, and never admit that his proposals also have cons. To be honest, I really think he believes what he says, he seems to not see ahead of time where the blocking points will be.
I am now forced to only submit the pros, and prepare an argument for the cons, which is rarely needed. This is not a way to have good technical decisions.
Good tests, I find, are to give them a simple piece of code with minimal comments. For example, some non-standard special purpose container implementation. Then ask them to explain what the purpose of the code could be as simply and short as possible.
Either - They'll start literally explaining each line of code without understanding the whole. - They completely miss the point. - They at least figure out it's a container of some sorts. - They'll explain technically what it literally does. - They'll give you the academic term for the concept it implements. - They'll actually explain the purpose briefly in simple words.
Those last three are good, in different ways. The better ones will also spot the bugs or performance issues in the code.
This is not how it works! There are over 1000 problems on leetcode. Sure, many of them are solved using the same techniques but there is usually a twist and you often need to combine multiple approaches or come up with something new (to you). And then there is usually a very problem-specific way in which the problem can be solved 10 or 50 times faster. Often these insights are very difficult to come up with. The people who created these questions are actually very smart and excepting some of the questions at the easiest level it's usually a far cry from simply implementing Dijkstra from memory!
The problem with leetcoding in interviews is that it is easily measurable but not an important metric. Focusing on someone's aptitude at skills they never use at their job means good developers get tossed aside or are forced to waste energy jumping through hoops needlessly.
In my company (totally remote) we decided to automate hiring (also remote) as most as possible and we are happy with the results. We also value the candidate's time very much so we made the hiring process very straightforward. It works like this:
- First the candidates must do some online tests, which are mostly multiple choice questions. The subjects are logic, english and a specific test for the coding language. All of these tests were hand made to us and customized to our needs. The candidates spend about 2 hours in total doing theses tests, and they know immediately their score in the parts of the tests that can be accessed automatically (which are the most part)
- The candidates approved in the theoretical tests - about 10% of the candidates that applied for the job - do a practical coding challenge that was also made from scratch by us. We have looked at the available solutions in the market and thought they are too focused on textbook algorithms that are rarely used in the job market. So we made our own tests that contemplate practical problems a programmer has to solve daily in our company. The questions are NOT too complex. The objective of this step is to weed out the candidates that can't code in the language, not to get the best. We design this challenge to take about 2 hours. The implementation was a lot of fun: we use the Docker API to run the candidate's code against the automated tests we made and the score is calculated as the number of tests that passed. Using Docker means we can work with any language that runs on Linux. So far we have made tests in Javascript, Typescript, Ruby, Postgres and Elm.
- The candidates that are approved in the previous step - about 2-4% of the candidates that applied for the job - are called for an interview. Now is the time when we take a look at the candidate's resume and look at the human side of things. Given that we have very few candidates to look at and we know they all meet some minimum skills we are very confident when conducting the interviews. By now we usually have a very clear signal to tell who the best candidate is even before we conduct the interviews. So in practice the interviews are NOT used to select the best candidate. They are there just to see if a red flag doesn't show up.
We are very proud of our coding challenges app. If someone wants to collaborate on it just drop me a message. It's a Rails app on GitHub, but it probably not good enough for an open source project yet - one needs to know a lot about Docker and Rails to make it work.
How many hours do you spend assessing each candidate and how does that compare to how many hours you make them spend jumping through your hoops?
But probably a lot of people that don't make it into your company just can't be bothered to do 2 sets of technical tests with zero human contact. I can tell you that my most "successful" interview processes (there is, the ones with offers/acceptances or just the ones I can even remember the name of the company) were the ones that focus on me as a human and as a professional, the ones that looked less automated and that seemed to have taken more time trying to know ME rather than just my skills.
Not that there was no tests, there were and some went kind of overboard with the complexity, but in the end what I remmember about these companies was the conversations we had and how they were interested and knowing me.
So, like I said, if it's working for you than great, but I probably would forget your test in my inbox to be honest.
Also, from experience, I've never come across a two hour takehome that actually takes two hours. Everyone always says not to spend more than two hours on it, but I can't exactly turn in an incomplete assignment.
I'm glad it is working for you, but it's not something to be proud of. Essentially what you've done is replace the standard 30 minute intro phone interview with a 2 hour SAT.
4 hours over two steps to get to the human part? I wouldn't mind it. Last company I successfully interviewed, I spent about 2 hours on the phone with different people + a lot of e-mail exchange, before I got to 4 hours long on-site.
That's asking for too much upfront before the candidate can even decide if the job is a good fit. Job descriptions alone don't tell you much. That's where the phone interview comes in. It's a two-way street.
I think when your company comes up with a hiring scheme, they need to analyze it, understand and be cool with the kinds of applicants their system is screening out. In my view, most SV tech companies' practices tend to screen out the "skilled professional who's currently semi-comfortable at her existing job" and filters in the "desperately needs a job and will jump through hoops and put up with whiteboard hazing to get one." Which class of applicant does your company want to court?
[^]: I consider myself "reasonably-competent" and the 100:10 ratio was about what my last job search was like a few years ago.
I understand this was the goal. You can either waste your company's time or the candidates' time. I don't think there's a way to preserve both. Given that your goal is helping your employer, this seems like the sensible thing to do.
If as mentioned previously wasting candidates' time filters out the most competent people, then you're not helping your employer.
Also, I object to thinking that someone's time needs to be "wasted".
Half the interview process is about selling the company to the candidate. How can I know if I want to put up with your bullshit tests if I don't know whether I'm going to enjoy working with you?
The message you're sending with more automation is simple: you will be a cog in a machine here. We're not interested in what you have to say, we're only interested in whether you're smarter than the average bear.
Programming or the fancy name of Software Development has sort of become the new blue collar job.
Easy. Just talk to people as would-be peers. They want to find a role they can succeed in. You want to find someone who can succeed in the role.
I interview by just talking about their past projects (based on resume). You can't fake your way through a friendly technical dialogue if you haven't done it. (Pre-requisite: the interviewer must be technically competent. This is where many companies miss. Algorithm puzzles won't make up for it.)
Also it is important to describe the role and expectations in detail. Nobody (really: nobody) wants to get hired in a role just to be let go in six months. There is no need for elaborate whiteboard algorithm dances which discriminate against everyone who doesn't have the time to memorize every possible question. Just be up front about the requirements and let the candidate decide if it's a match. They know themselves better than you ever will.
What troubles me is that people will say this allows all your biases to run wild, but my feeling most of the time is that these biases will find a way to express themselves unless you remove humans from the hiring process entirely.
You can’t pre-correct for some things, but you can for others.
Having someone speak in detail about their experience isn’t easily faked.
Having someone speak about their experience in brief and then giving them a “skill-testing question” as if they won the lottery can be gamed.
I absolutely disagree with this.
I've interviewed people who seemed incredibly impressive, both in terms of showing me actual past projects, and in their explanations of them. I've then gone on to give them simple programming exercies, the kind that people with far less experience take, and they've gotten nowhere on them.
I'm talking a two hour, "here's a computer, here's internet access, here's a setup environment, code this simple thing, which you could probably copy-paste from Stack Overflow / documentation". And they couldn't do it.
And what's worse - I'm pretty sure I could fake my way through most technical dialogues. I mean, not literally everything, but I could probably fake my way through interviews about most technical topics, and given a bit of preparation time, fake my way through most things. I mean, I could probably get caught out if someone was seriously trying to do it - but to the depth that most interviewers get, I can pass as experienced on most things.
I feel "just wing it" can lead to somebody talented from a non-traditional background getting a chance at a great career that they would be auto-rejected by google at the resume submit phase.
Which seems reflective of the underlying pool of applicants.
It would be interesting to see what their application/success ratio demographics are like.
1. http://www.bayareacensus.ca.gov/bayarea.htm 2. https://www.wired.com/story/googles-employee-diversity-numbe...
Of course the most important thing is: out of minority applicants who are qualified, what percent are hired? Because we may expect a smaller proportion to be qualified due to systematic barriers that they face. Google is not necessarily in a position to fix those.
How many of them hold relevant degrees for job openings at Google?
There's even a sourced 2018 article about current hiring practices trending away from degree requirements for you to peruse:
https://www.cnbc.com/2018/08/16/15-companies-that-no-longer-...
It's not about the degrees (relevant or otherwise).
I'm willing to bet that Google's demographics hew pretty closely to CS graduate demographics. You need to look farther upstream.
And for the social justice aspect, we do have a very diverse group of engineers, especially for our size and location.
You want your company to be 50% women? Tell your teams that at least 50% of their hires must be women. That's it.
To me, formalizing the quota but keeping the overall hiring process looser makes a lot more sense than the other way around. A quota is dead easy, and a formal process is hard for all of the reasons discussed.
A quota is not "hiring people just because they're female/black/gay/etc"—it's a way to make us more effective by preventing us from overlooking good people. If we're naturally inclined to skip people who are black, then we need a process to counterbalance our natural tendencies. And of course, recognizing and taking advantage of an undervalued resource is good for business.
If there are legal issues with a quota (I'm legitimately not sure whether there are), then the law ought to be changed.
---
50% women may be the wrong number because of the demographics of computer science majors or what have you. It was just an example. Choose a number in good faith that you believe represents the potential population of good hires.
One simple solution might be to have your quota match the demographics of your job applicants. But this decision bares more thought than I'm qualified to provide.
So, while I wholeheartedly agree that diversity inherently benefits teams and businesses (different experiences will lead to new and different approaches), it's just not a very convincing argument, because it feels "mushy". Here's one that I think is better:
If it's true that e.g. women have a harder time getting engineering jobs due to unconscious bias (and I'd posit that this is clearly the case), then that means female engineers are, as a group, undervalued. You should be able to hire a better female engineer for the same amount of money.
Harsh? Sure. But that's beside the point.
Cite this.
For the tech sector, in developed nations, this is nonsense.
The evidence for that is dubious
1. I hit the wrong button :)
2. I'm a bigoted member of a majority, and am downvoting because I think minorities should be oppressed.
3. I don't think any questions of race/gender/etc. should ever be considered in a workplace, and and downvoting because I dislike the whole idea.
4. What I suspect is actually happening: Like (3) I think everyone should be considered on their own merits, and you can look at company-wide stats to see if bias is occurring. Its good for the company to act against it in sum, but acting on the basis of race/gender/etc. on an individual basis is shocking and unfair.
The problem with (4) is that you're trying to have some kind of emergent effect that magically appears. It's a goal without being a goal. Worse, its a global goal you want to be actively worked against locally.
Developers love to complain about clueless management that doesn't know what it wants. Guess what folks, it's a common trait. Our other favorite trope is to figure out a technical fix for a people problem. And the problem with that is of course a tech fix only takes care of specific situations and not the underlying problem.
For example, as a tech fix we might decide that all interviews are done in a pitch-black room with vocoder voice synthesizers. Great, no question of race/gender/etc. Except, of course, for selection bias in our candidates. And even after they are hired there's biased reactions (unequal success for equal quality). People just naturally favor familiar attributes, even if those attributes don't contribute to a correct outcome.
So to be consistent, there are really only three choices: a) don't care, let the chips fall where they may; b) have an actual goal to equally represent everyone; c) encourage playing nice, and hope the results aren't too far off.
What actually happens, of course, is d) encourage playing nice, hope the results aren't too far off, then someone actually looks at the results and complains, so flail around trying to avoid doing anything too unpleasant to fix it.
That would be quite literally destroying the anti-discrimination laws which safeguard the exact thing that you think you're arguing in favour of. Quotas implemented the way you describe are the antithesis of equal opportunity. That 'instinctual negative reaction' you come across is people seeing the complete logical reversal going on in your claim.
Not only that, but it's blatantly hypocritical since you don't spend your money based on the demographics of a company, you spend it based on performance.
Edit: This leads to a much more interesting line of discussion that can be summed up as "discrimination (and similarly bias) isn't always bad".
They're useful tools and we should be careful about practicing them, but the idea that because discrimination is sometimes bad it is always bad doesn't follow. (or an alternative way of looking at this is that discrimination often includes the qualifier "unjust". Bias in favor of some group in the pursuit of greater justice therefore isn't discrimination).
I think most people making a case for 'good' discrimination don't realize that the existing 'bad' discrimination originated exactly the same way.
We need to strive to eliminate discrimination, not perpetuate the swing of the pendulum that's been going back and forth for hundreds of years. Pushing down on some demographics to uplift others is not going to solve anything, it's just a no-op perpetuation of what's already going on.
The historical discrimination did not originate from people saying "yes discriminate against me in the interests of general equality". In fact, to return to the start of this thread: historically much discrimination was the result of bigotry. It's very unlikely that hiring quotas (or similar) are the result of bigotry.
Your conclusion also doesn't follow at all, at least not without some argumentative work that you haven't presented. Suffice to say that I believe that the best way to, in the long term, eliminate discrimination is to allow some specific forms now, with the intent of undoing past discrimination.
You haven't presented even a shadow of an argument as to why that isn't reasonable.
The way to create equality is to manipulate on levers that individuals can control, e.g. taxation brackets, education incentives, government investment into particular industries. Passing up the best candidate for a job because they didn't have some inherent unchangeable attributes that you're arbitrarily favouring is the opposite of creating equality.
Genuinely asking. My morality follows from basically “discrimination based on immutable personal characteristics is always bad” (with the possible exception for family, bwcause that’s just cruel), from which I cannot conclude for example that discrimination is bad if it’s agains blacks but good if it’s against whites.
I fear that by dropping this principle, while allowing for “positive discrimination” (e.g. preferential hiring / admissions of blacks/women), you also allow e.g. White Nationalists to argue that it’s OK to discriminate for whites. How would you convince a White Nationalist that such discrimination is bad while leading by example that other kinds of discrimination are just fine?
> My morality follows from basically “discrimination based on immutable personal characteristics is always bad”
As a bit of an aside, are you saying that this is your base moral precept from which all other decisions about what is or is not ethical follow? This would be a very strange ethical framework (for example, it follows that murder is ethical). Usually moral and ethical frameworks follow from very high level principles "optimize happiness/utility" (utilitarianism), "do good things" (deontology), etc. It seems like you might be coming from a rights based ethical framework, but it's not clear.
> How would you convince a White Nationalist that such discrimination is bad while leading by example that other kinds of discrimination are just fine?
I don't generally waste my time arguing with people who are engaging in bad faith[1]. In this day and age, I don't generally believe that a white nationalist is going to be convinced by facts and logic, or really anything else. I'm more concerned about appealing to the majority of people who aren't white nationalists, because ultimately I believe that discriminating against white nationalists is okay[2].
Ok, now back to your actual question: in what cases is it okay to discriminate? I'm approximately utilitarian, so the simple answer is "when the expected value across society is positive." For me, I don't think optimizing for strict utility is the right goal, instead I personally think that we should maximize agency (possibly median, possibly total, unclear exactly), which is something like each individual's ability to make meaningful decisions.
This sidesteps issues of de jure rights. It leads to conclusions that we should do things like lift up the least fortunate and provide social safety nets, since a destitute person has less agency than a person capable of getting by. Similarly, it addresses questions of when regulations on personal liberty are ok. If allowing some action has a reasonable possibility of reducing agency of some other group, we should give it less protection than another action which does not.
So for example, when you have two groups, racists and some racial minority, it makes sense to provide the racial minority more protection than the racists, since the racists want to reduce the racial minority's rights more than the racial minority wants to reduce the racists rights. That is, the minority would prefer it if the racists weren't allowed to be racist. The racists would prefer it if the racial minority weren't allowed to live in the same area, or use the same water fountains, or what ever.
That's an extreme, but illustrative example. So back to the less extreme position: Discrimination is okay when it's done explicitly to counteract a gap between what the law (or more broadly society) intends and where society actually sits.
Society intends for hiring to not discriminate based on race. But it did for a really long time. The put (and continues to put) the subjects of discrimination at a disadvantage. If you're deontological, then "nondiscrimination" is the rule and that's that, but if your goal is justice or utility, then discrimination to bridge the gap between intent and reality is ethical. It should be done carefully of course, but it's alright.
If you want to see a fun set of examples of this, voting district and gerrymandering law in the US is full of weird examples of trade offs between the need (under the Voting Rights Act) to allow racial minorities to elect representatives of their interests, with the current opinion of the USSC that any districts can't be drawn based on racial lines (except in very specific circumstances)[3]. In other words, States must draw districts to preserve the interests and representation of minority groups, but can't draw districts based on where those minorities live. It's weird.
[1]: https://en.wikiquote.org/wiki/Jean-Paul_Sartre#Anti-Semite_a...
[2]: https://en.wikipedia.org/wiki/Paradox_of_tolerance
[3]: https://www.ncsl.org/research/redistricting/redistricting-an...
> If you're deontological, then "nondiscrimination" is the rule and that's that, but if your goal is justice or utility, then discrimination to bridge the gap between intent and reality is ethical.
I'm definitely pro-justice, it's just that I see it principles based (e.g. free speech even for (especially for) speech I don't like), not "utility" based. The flaw of your philosophy, in my view, is that you're just sweeping the tricky parts under the carpet by introducing concepts of "utility" and "right goal" and "society intends". You're even conflicted about that yourself, introducing another concept of "maximizing agency" (sounds like pro-freedom); a good moral system would somehow maximize (or balance) both in some way.
I don't think that's a good view, because those are fundamentally non-reconcilable goals. Should a billionaire be able to spend $1m for the health of his/her child (maximize agency / freedom), or should the society put that money to "maximum utility" and spending it to help several (poorer) children (with illnesses that are less expensive to treat but still life threatening).
Sure, picking the "right" utility function is one of the hardest parts of utilitarianism. FWIW, I don't think maximizing agency is a different concept so much as a specific utility function that avoids some of the common pitfalls of strict utilitarianism (utility sinks: the person who gets enormous happiness from cannibalism). In other words you could imagine Agency as the "reasonable person" standard applied to the value of a choice. This still has flaws, but, IMO fewer than many other framings.
Some people might also find this to be similar to optimizing for the most "positive rights" across society, but I dislike that framing because the idea of positive vs. negative rights is silly, any right can be framed as positive or negative.
> Should a billionaire be able to spend $1m for the health of his/her child (maximize agency / freedom), or should the society put that money to "maximum utility" and spending it to help several (poorer) children (with illnesses that are less expensive to treat but still life threatening).
Note that giving a poor person money such that they can choose to treat their child's disease also increases agency! If you want to dive deep into this, I think that the marginal value of a dollar is probably logarithmic, or at least inverse quadratic or something, so a tax scheme modeled to tax top brackets in such a way that then improves the situation of those less off (pick your method: national healthcare, UBI, etc.) increases overall agency.
Profound injustice in favor of averages is not an acceptable social trade off. This is why countries enact general discrimination laws, making it illegal to judge people based on single bits like gender and race.
For example, in most places in the US it's okay to discriminate based on political party, and I don't know of any jurisdiction where you aren't allowed to discriminate based on someone identifying as a racist.
Political affiliation, political views, political association, all those are protected.
The state can step in if it is in the interests of national security or to preventing of crime, but it is just as wrong to discriminate someone who identify as a feminist as someone who identifies themselves as a racist.
It should also be noted that where I live, Sweden, the right of political association is particularly cemented in our constitution. The same protection that is given union members are given to political members, including racist political groups. People talk about changing the constitution whenever neo-nazists goes around and demonstrate, but that kind of talk usually dies down when people realize that life goes on even if a handful of people believe in something which the rest of the nation consider crazy.
It appears that you're conflating government nondiscrimination with government mandates on individual nondiscrimination. Those aren't equivalent.
Here in Sweden, yes. The law is very clear that you can not deny service because you disagree with someones political, religious, or world views.
Government nondiscrimination is a US technicality of the first amendment. European Convention on Human Rights is not limiting to government behavior. Everyone has to follow it.
I think you should speak with a lawyer, because you're not correct.
The report also note that current law does not protect children that get discriminated based on their parents political activity, which the FN convention of children rights says they should be protected against.
The report goes on to basically say that the EU convention of human rights extends the Swedish anti-discrimination law to include, among other things, political and other world views, class, and social status. The report then goes on that courts must already follow those extensions.
There were also a specific case a couple years ago where a politician was kicked out of a restaurants based on his political affiliation. A Judge commented on that case noted that while the Swedish anti-discrimination does not make it illegal, there is a case to be made under the EU human rights declaration.
So there. The government report, and a judge, both saying the same conclusion.
This is a subject of entire books and hours long debates, but we can skip to the reductio ad absurdum with your line of reasoning: 99% of underwater welders are male.
Trying to 'represent' females in that occupation and push the gender ratio to 50/50 would harm all parties involved and result in a measurable number of deaths. Protected attributes are not uniformly distributed in terms of fit for job, and trying to hit a uniform quota across those attributes is naively idealistic, at best.
A less generous interpretation is that you are actually totally cool with discrimination as long as it's in the 'right direction', whatever that personally means for you. In which case you are exactly the force that you claim to be fighting against.
The notion of reverse racism is something that requires religion-like amounts of self delusion and selective consumption of facts. The flaws of the notion are just too glaring for any rational individual to overlook even at first examination.
I do not know what the costs, efficiency and employee happiness would be, but it would make for an amazing social experiment. Having 40% of the garbage men, sewer workers and construction workers being women, and 40% of the teachers, nurses, and social workers being men would make for a very different society than what we have today. There would be a lot of people trying to cheat the system in order to keep the work place gender segregated, but I wonder if that is a local maximum of happiness rather than a global one. Such study could shine some light and give us some final answers if quotas is a good idea or not.
(While there's a separate discussion to be had about why so few women choose to be engineers, that's not the company's problem to solve.)
There's no logical reason why "10% of engineers are women" implies "10% of every engineering team should be women" instead of, for example, "10% of engineering teams should be all women".
A number of teams could be 40% women even if the average were 10%, and there's also no reason why excess men have to be engineers just because they were educated as such. That's a thing that happens anyway, just like people without CS degrees go into programming.
Standardized processes don't mean unbiased processes. The SAT for example is pretty racist.
What do you mean by this?
Is it equally racist now versus in the 90s?
Depends what you mean by "use your gut" but that's not my experience. Currently at a well known public tech company in silicon valley and there isn't any standardized interview process. Every interviewer is free to approach it as they wish and the hiring manager gets to listen to all input and decide. This is a strength.
Perhaps (very likely) as a result, this company is also rated highly in diversity.
A standardized interview process will inevitably lead to a standardized employee pool, as only the candidates who fit the very narrow profile (willing to memorize endless algorithms they will never need or use in real life) can get in.
I should add that my current employer is not, in my experience, unusual. In my ~25 years in the valley, it's how it's always been. I have not worked (and never want to) for the companies which value conformity in hiring above all else (infamously, google).
If so, how are they "objectively" evaluated?
Do chemists recite the periodic table backwards in interviews?
I don't pretend there's never any bias, but I'm still not convinced you get any less, or any better bias with the whole hoop-jumping algorithm day-long whiteboard thing.
As has been noted elsewhere in the comments here, ours seems to be the only industry that does this to itself.
What did they do? Suggest sacrificing a puppy to moloch to make a feature release go smoothly?
https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
Is the amount of youth in tech part of the problem?
On the other hand, another company I applied to that seems really desperate for developers (no fewer than 9 open dev positions when I applied) completely dropped the ball. During the screening process they asked me to do an online test that took me about 2 hours. I got 100% on the test but it took the HR rep an entire week to respond. The response? The same canned email I received when instructed to take the test. From the same person. After a back-and-forth it was confirmed that they did receive my initial test score and I didn't need to retake it. About a month later and they never did get back to me. Sometimes I wonder if companies even want to attract good talent.
Sometimes we find a unicorn and want to make an offer. They get hired by FAANG after our interview and before we send the offer. Wonderful.
Contrast that to my side hustle where I sometimes hire folks to help out with coding so I can focus on other stuff.
Go into 1 or 2 Slack/Telegram groups "Hey anyone need work?"
Exchange 5 texts with 3 people.
"Okay here's a project, how much?"
Pick person. Get job done. If I like the way it goes, they get more jobs later.
Whole process from search to hire takes 1 week.
But at the dayjob we're a funded startup so we gotta be choosy and picky and make life all sorts of hard in the name of I'm not even sure what. We wouldn't even consider a part-time contractor because commitment is worth more than getting the job done.
¯\_(ツ)_/¯
If you do good enough work at a job that your coworkers will want to work with you again, you've got it made and can choose between jobs for the rest of your career.
If you don't impress coworkers that way, you need to go through this crazy stranger interviews talked about here. Do try hard to avoid that!
In all the companies I've worked for, referrals for ICs meant that these people were prioritized for interviewing, not that they skipped the process. And no one, not even the manager of the hiring manager, had enough say in hiring someone that they could simply decide to skip the process
Companies that really want to hire are flexible. Maybe they need someone to do Python but they've found a good candidate who has all of the other requirements except for Python - but they happen to be really experienced with Ruby. Well, OK, if they're really experienced with Ruby they'll be able to pick up Python in a month or so. Sure, they may not produce the most pythonic code for the first six months, but that's what code reviews are for.
I've been in this game since the 80s. Back then companies weren't nearly so picky it seems (at least not in my memory). The second job interview I had out of college in 1986 at a startup in the valley went like this (it was an embedded/hardware job):
Interviewer: We just got this new CAD system. I'm going to have you sit down and play with it for several minutes. When I come back I'd be really interested in hearing your opinion of it. What are the good and bad parts?
And so I play with the CAD system for a while and when he comes back I give him my opinion. And he agrees with my assessment.
Interviewer: I've got this schematic here and a breadboard, some parts, power supply and test equipment. Do you think you could hook the circuit up while I attend to some other business? I'll come back occasionally to see how you're doing.
I wire up the circuit, get it going, look at the output on a logic analyzer. Interviewer comes back and seems happy. After this he asks when I can start.
This took all of maybe 2.5 hours. In the 34 years since then I've only had a couple of other interviews that seemed that... I dunno how to put it, maybe "natural"? And both of those were in startups. They wanted to hire someone. They needed to hire someone.
Back in 2001, I dropped out of tech and became a land surveyor for almost 10 years. Now, land surveying or civil engineering is not the same business at all as software development, but they are both technical and there is some overlap in terms of engineering and outcomes.
My technical test and interviews were all completed on the same office visit, and I got a decision in 24 hours! Just think about how incredible this would seem today. First, I was welcomed by an engineer who gave me an overview of the company. Then, I was taken to a conference room and given a trigonometry & statistics test that took about 45 minutes for me to complete. Then I waited in the engineering library and browsed books while they graded my test. When my results came back, I then proceeded to the interview proper. After that I was introduced to the CEO of the company and escorted to the lobby.
The very next morning, I got a FedEx envelope with a proper offer letter. They really needed a good instrument man and they got one in 24 hours. Throughout the entire process, everyone I interacted with was completely professional and prompt.
This means one phone screen of about 30-60 minutes to both introduce the candidate and decide if they are worth a follow-up with the team followed by a one hour interview with the whole engineering team more focused on technical capabilities directly related to the job. Finally, just to be sure, a one hour take-home assignment that is simple just to make sure they are not bullshitting. This last one only goes to the top few candidates of which one will be chosen (assuming they can code the simple assignment). People have been hiring this way (ok minus the coding assignment) for decades, maybe even a century or more. And you know what? It works.
Now on to the second requirement: having a company that's worth working for. That means a company that at least partially values its employees. The best way to do this if you don't have FAANG money? Remote work. Ideally for everyone. That guarantees your pool of hiring (even if limited to certain locations for artificial reasons) is much higher and the demand for the job will be huge. Other things that make a company worth working for: good salary, yearly raises, performance bonuses, 40 or less working hours a week, flexible schedule, decent managers who understand engineering, interesting tech, etc. Note that none of those is out of reach of any company, even the smallest small-business.
So yeah, let's stop with this bullshit that we don't know or can't think of a better way to hire. That's idiotic and frankly, closed-minded. Let's stop pretending like there aren't a ton of qualified candidates just waiting to be hired when there are so many. Let's give people a chance for once to prove that they can do the job. Not everyone will work out, but you know what, in the US with at-will employment, that is not a big deal. Doesn't work out, fire them. Easy fucking peasy. If you can't do that, you shouldn't be leading a team/company.
My company currently works this way and we have had NO problem hiring. Whenever we want a great candidate, we go from resumes to hired in less than a month. There is nothing special about us. We don't pay top dollar. We don't even have a lot of the perks I listed above, but we have enough of them that candidates love us and we have no problem attracting great talent (the top one being remote work, of course). If a company isn't willing to make itself desirable, I have no pity for them. It's like never showering or grooming and expecting to find a hot girlfriend. Highly unlikely.
A month is considered fast?!
I guess as long as the great candidate you're hiring is not looking at other options.
A recently saw a recruiter lamenting that their clients needed to be told that if they don't make a hiring decision within 2-3 days of introduction, they tend to miss out on the best engineers, who get offers from elsewhere within the week and take them.
If you have a steady stream of equally capable, qualified candidates, it's a buyer's market and you don't need to do anything. Carry on :-)
There are other companies who complain that they can't find qualified candidates though. Some of those have those positions that stay open for months.
Those are the ones the recruiters urge to change practices so they have a chance at getting "the best" engineers, on the rare occasions "the best" interview with them. The seller's market situation, where buyers need to compete.
It sounds like you don't have that problem though.
> I tried explaining that perhaps that's something you shouldn't expect from someone who's at that point switching jobs.
Ugh. You are, of course, correct.
They want "the best candidates" but when they get one that's evaluating new positions they go on about "why do you want to move", "candidates don't seem to enthusiastic about our company" well maybe because you're not enticing then to move, quite the contrary.
Good candidates get hired (from an unemployment/"underployment" pool), excellent candidates get poached. Sports teams gets that but apparently if it was a software company they would ask Michael Jordan if he would be able to hit a 3 pointer
Then I recently had an interview myself (thank god for no pressure) where I was asked to do more or less the same thing myself, and I totally blanked. It took that to make me realize it probably wasn’t a good problem.
At the same time, pretty much all the candidates we’ve hired have worked out.
I’m starting to think it might be easiest to just hire anyone that applies and doesn’t seem crazy.
I had a great job lined up, perfect fit for me and them. But my SO has a condition. The pill my SO takes everyday costs ~$1-5 per pill with most insurance companies. The substance is used in horse feed, just like how humans put vitamin D in milk, or iodine in salt, this substance is put in horse feed gratis. It does not appreciably change the cost the feed.
When I looked into the health insurance for the new job, the pill would have cost us ~$100, so ~$3k/mo or ~$36k/year. I, obviously, had to turn the job down.
That US healthcare is completely broken is not just contained to families, but it leaks out over our whole society. A perfect hire, ruined because of shitty healthcare.
I've outlined it here before. Been using it and tweaking it for the last 3 or 4 years. I keep bring it up because it's a tested alternative to the usual death marches and dumpster fires and it has been a success for the teams I've led and our applicants.
The principles I've defined to guide our process[0]:
- Hiring cycles will be structured and as short as possible.
- When we start a hiring cycle, we will finish it by hiring the most qualified applicant who accepts our offer.
- Every applicant will receive a response within 48 hours and be updated on the status of their application at each step asap.
- The hiring process will be as transparent as possible.
- Objective and fair-minded measures will displace biased and bigoted ones.
- Every applicant will appreciate their experience, even the rejected ones.
- The process will be agile and adapt over time to improve and meet the specific needs of the organization.
- Onboarding will begin with hiring.
One problem I'm confronting at the moment. As our company grows, we've brought on a new HR lead who wants to jam the process we've been refining over the last couple years into a new ATS (Applicant Tracking System) that could significantly screw up our meez. We'll see how that works out. I'm doing everything in my power not to let it fall apart.
If you have the power to improve the situation, please do so.
[0] For an outline of the process, see https://wiki.klenwell.com/view/Hiring
Most of our work is on web applications, primarily Rails. So, for that role, the challenge is to fix an issue in an implementation of Fizzbuzz on our sandbox Rails app. We give the candidate about an hour to work on it. It's not as trivial as the common Fizzbuzz exercise but it's a pretty representative sample of the kind of work we do.
As part of the phone screening, we check to make sure the candidate understands there will be a code challenge involving Rails and that they're cool with that.
There's a handout we present before we turn over the laptop that explains the issue and clearly defines requirements (e.g. create a branch with this name). We read a brief intro section then give the candidate time to review the requirements on their own and ask them if there are any questions. Candidates are reminded to read the README included with the project and encouraged to use the browser to Google stuff.
One tweak we made was to add check-in stages to the challenge to make sure candidate doesn't get stuck for a half-hour trying to open up the text editor or something. Every 15-20 minutes, a team member will check in with candidate and if they haven't hit a certain milestone (e.g. has started local server or had reproduced issue), we'll assist them in getting to it. At the second or third check-in, it usually turns into more of a pair programming exercise and at the end we'll review the solution with candidate so they understand why the problem is relevant to the job.
There's not a definitive pass or fail for the challenge. The candidate's performance gets scored on a simple 1-5 scale alongside the other competencies we're evaluating as part of the interview. We're pretty thorough about preparing the code challenge and consider it a good example of our "onboarding begins with hiring" principle.
Even sticking to the pure 'trivia' aspect, there's just so much of it that people are missing altogether. (Sure, maybe you can write out a nifty algorithm on the whiteboard, but can you sketch even a very rough proof that the algorithm is correct? I don't think many people can do anything like this. I'm not even sure that the topic of correctness comes up all that much in a CS/SE curriculum - which is surprising, since we are supposed to be doing engineering. And yet, someone who is familiar with what provably-correct code might look like in practice will likely be a lot more productive as a result.)
I decided to not to do any whiteboard challenges unless the whiteboard has a built in editor, search engine and stack overflow. So far people who interviewed me just wanted to know about the different projects that I worked on during the last 15 years.
I'm lazy and don't have time to prepare for interviews or do home assignments because of family life. I'm going to be frank about this if I'm asked to do anything more than talking during an interview.
People blindly and dumbly implementing any practice is going to yield bad results. I've interviewed for Google, for a small company with a crappy pipeline and a low need for talent, and for my current job, which needs Google-level talent across multiple specialties[1]. Whiteboarding has played a critical role in all of these. The goal isn't to have them answer trivia questions in marker, it's a basic test of their ability to think critically and program. Naturally, the exact questions differ based on the role: for the second job, it was basic processing of data structures, while for the third, I got a question that involves some degree of spatial reasoning. But in general, this is a pretty fundamental way to test a pretty fundamental set of skills.
I even was lucky enough to get a negative example: I ran the tech org for the second company, which had two non-technical founders. We hired a candidate with no engineering experience on my recommendation because he did very well in his coding interview: this guy ended up being the most reliable, clutch engineer in the company. I also rejected a candidate on the basis of failing the simple whiteboard problems, and the founders decided to hire him anyway based on his years of experience. The founder ended up having to bounce him around from task to task, leaving a trail of destruction wherever he went. His code would bring production down, he'd create weeks of work for others in a couple of days, he utterly destroyed devops during his brief tenure there, etc. He was very sweet, hardworking, had a great temperament... But the inescapable fact was that he was just kinda dumb. The whiteboard interview handily caught this, _which is what it's supposed to do_, and all the HN sniveling about whiteboard interviews only being useless trivia contests doesn't erase that fact.
[1] You can imagine how hard hiring is for us right now...
OK, but this topic is not about "simple" whiteboard problems. (People can of course disagree wrt. what's 'simple', but a rule of thumb is that anything where the average dev would need to train and memorize whiteboard trivia for an extended period in order to perform satisfactorily is far from simple.)
I was talking about minimizing risk for the people hiring at sub-FAANG companies. If they make crap hires, they can just say "well we're doing it exactly like FAANG". But yes, I also agree there might be better ways to optimize for minimizing bad hires.
Despite it being a turn of the decade 1980/1990 Microsoft thing.
Meanwhile asking questions that are grounded in the actual work you're going to have the candidate do, often gets you people making the estimation even if they haven't heard of Fermi Estimation, ever.
That's what makes them riddles, to me, and mostly useless BS in actual interview.
When someone says they hate "logic problems", it sounds as though they're talking about this: https://www.pennydellpuzzles.com/products/logic-math/origina...
I don't like those either, but I doubt they are used in interviews.
Many common examples of interview riddles that are defended as "it's just Fermi Estimation!" end up "memorise, interview, forget" method for many people, because there's close to total disconnect between assumed background knowledge and actual background knowledge (a question I encountered recently asked for estimate of revenue of an MLB stadium. If not for being spelled immediately after, my questioning would start with "wtf is MLB?").
Then there is the other part, namely that unless you don't know (memorise) certain common tropes of a Fermi quiz, you might understand the question as asking correct answer, not showing that you can use random number for most of the input data.
That said, I have been asked things that are fermi estimation problems on interviews that didn't suck and didn't feel like pointless riddles of Mensa wanna-bees. They tended to be based on the work problems, thus allowing to have a real and concrete expectation that the interviewee shares the background knowledge (in fact, part of the reason for the question might be checking if they do share that knowledge!). That kind rarely gets the flak of "how many piano tuners in city of Chicago".
The fact that not everyone knows the same things is the whole point! And if you don't have any knowledge that is related to the interviewer's, then how are they going to evaluate you using any method?
But sure, there are specific, inflexible requirements. You have to know false precision is BS, and I have no sympathy for people who have a problem with that. You have to in general, know what you know and what you don't.
OTOH, especially if you're interviewing internationally, using "common knowledge" instead of job/technology area related one? Pretty much a fail, helped along with books dedicated to memorising bullshit questions.
Another issue is that the typical "fermi problem" is stated in absolute, precision way. Ask "how would you estimate X" and you'd probably get good answers even from people who don't memorize interview gotchas.
But I suspect the reason people widely hate the concept is because schooling pounds into you that there is only one way to get the right answer, and it must be exact.
And so when people complain about being tested on trivia, what they really mean is "I want to be tested on trivia that I know". I would actually like to be evaluated based on something other than knowing trivia, but of course I would not like to take a stupidly constructed test.
I used to take interviews and I always knew the outcome. "Sorry you are self taught, we only want people with CS degrees." "Or you don't know how to reverse a list." "You don't know what a bubble algorithm is" Why did I keep punching myself in the face?
Well I decided to make a change in the last 6 months, I wanted off the hill, the ski lift or what ever ski metaphors your want to use. It's summer and I'm going surfing. (also we both live in Colorado, it's getting little crowded on the hill and the traffic to the slope is bad.)
I changed my attack, why was I interviewing for these horrible jobs that I really didn't want anyway. So I changed my pitch.
I now write algorithms for futures/stocks/options, I work for a company that have a ownership stake in. My code is good, but it might not perfect. No ego or code reviews. Because the true worth of my code is not how many classes I use it whether it returns profits. I didn't get into coding to so I could debate OOP structure or MVC models. I write code so I can make a ridiculous amount of money. Nothing more or nothing less. The code is just means to make more money.
Enjoy the lift ride, but I'm going to Hawaii.
About a year or so I decided it was time to get back to my financial knowledge of markets and my ability to program. I need to get out of the trap of a dead end programming job. (I was working in Drupal 8... it doesn’t get any worse) I already had one stock algorithm so I started working on an options algo. Then via the internet I ran across a family office that needed assistance with a futures algo. I’m luckily because I have strong academic understanding of financial markets and the ability to build algo and write code to api’s.
Sure if you want to work in a co location, high frequency trading firm you might need a Harvard degree, but there is no reason that you can’t write your own profitable algo.
(this is the Portfolio button URL in the nav bar)
We tend to praise people who take the risk, fake it till they make it are go getters etc. And possibly these are trying for that.
And maybe it would be pretty cool to see a sociological study about who these people are. What demographic they belong to, what background they have, what kind of personalities and value they have.
One issue is that average management is not good and management of software projects is not solved problem. That badly affects hiring and promoting in many teams.
But another issue is that tech is wide range of jobs that require completely different aptitudes and skillsets. We are not even starting to classify it all enough to be able to reasonably communicate what position is.
It seems implausible. It's counter-intuitive. But the idea seems pretty persistent.
It's an interesting question for me personally. I'm a mechanical engineer who likes to program a little and I've sometimes wondered whether I could work as a programmer. I haven't got the right qualifications or experience with enough popular technologies, so moving fields would be tough. But I know I can implement basic things from scratch. It doesn't seem like much (and clearly it wouldn't be enough for the prestigious places with really tough computer science interviews) but it would be interesting to know if that actually was enough to clear a certain low bar of competence.
Some of us have direct experience with exactly this situation (see my previous comment). Hell, Jeff Atwood wrote a whole piece on it a long time ago.
Further, most interviews I encounter don't do this and the ones that do tend to be all goddamn cagey about what to expect so I never know in advance, so I don't have a home whiteboard to spend hours and hours practicing on and if I did I wouldn't know which interviews need such prep and which don't so I'd probably end up not bothering, since it's just a few that do it.
I don't even have significant social anxiety or anything, and I have been told I come off as very confident and competent—unless the whiteboarding algo shit comes out, at which point there's a 50/50 chance I'll look like a total dunce (Nb. I can and do get The Actual Work done, no problem, once hired—that part's easy)
Since I only interview every few years and, again, this problem hasn't stopped me from landing jobs at about the same pay rate as the other ones that rake me over the coals (though not FAANG or near it) I've not spent the significant time it'd take to fix this. We'd be talking lots of hours to prep for FAANG tier, and probably a year. Don't care enough to do that. I did spend the time & effort to improve how I come off in more conversational parts, gradually, over the years, but that was relatively easy and doesn't rust as fast (since, you know, I don't get to practice interview-type coding on the job, but I do frequently have opportunities to practice relating to people).
[EDIT] I don't mean to be whiny or call the grapes sour—I simply haven't put the work in to (as I posted elsewhere in the thread) turn my efforts at leetcode from practice to performance. And again, for the roles I'm applying for I'm not losing any money by occasionally being rejected because they decided to give me a puzzle expecting a recursive & memoized (DP) solution rather than ask me whether I know WTF recursion and DP are and when they're usually useful, so the worst they do is waste a day of my time (honestly, why the hell are places so secretive about what's going to happen on interview day? Fucking Google's not, Jim Bob's Software Widget factory can afford to tell applicants the sorts of things they'll be talking about and the kinds of exercises that may come up)
Anyone who cares about using the wrong stdlib function is bad at whiteboard interviewing. That's exactly the kind of thing that's easy to mess up in an interview, as well as completely irrelevant to job performance.
I've been writing in C++ for the last couple years, and I'd probably screw up writing a new class from memory 50% of the time.
Interviewing is attempting the ridiculous feat of predicting years of job performance on the basis of a few short conversations, so there are going to be false negatives in every interviewing technique. The things I describe above lower FNs to the point that whiteboarding is a very useful tool for assessing candidates.
"Dumb people work in software eng" is true, but it isn't a novel or useful insight; you may as well complain that cloudy days exist or that closed systems tend towards entropy. It's the reality that every industry and business operates within, and the claim that it's unique to software is wrong-headed.
Half my comment was about the fact that _every_ field has these idiots. And by what bar are they "not adequately trained"? If someone wants to hire a stack overflow copy-paster, let em. I'm sure their productivity is strictly nonzero.
If the 'people fail fizzbuzz' trope is actually true then you would probably clear that first bar easily and probably the next few as well.
Personally, I think most if not all software devs have some form of tunnel vision (aka limited experience). While they feel the bar they are setting is low, the manner in which they describe the bar may be completely unfathomable to a person not familiar with the same terminology.
Since all technology is just new terminology for old ideas then it's easy to fake competence if you know the new terminology or look like an idiot if you aren't a slave to fashion.
That cycle has been going on for years. Now, there are so many companies that are essentially writing variants on the same software over and over again that demand is dictating the shape of the field. The shape of the industry. “We” are all industrial designers in a world that just isn’t that enthused.
If we wanted this other world that we keep interviewing for, then it’s not a matter of not enough programmers, but way too many. They’re cheap enough that people everywhere are reinventing wheels instead of assembling off the shelf components designed for that purpose.
People who see their jobs as safe careers are routinely good at those jobs.
And conversely, I have met people who talked about passion and seemed passionate and had all the right opinions. And they were not good simply because they would loose interest when it stopped to be fun or somehow never finished anything that was medium sized, were disorganized or demotivated easily.
I speculated on why our interviews are so insane, and just kept pulling that thread. What I omitted from GP was this thought: I don't know if we are right to want this, or if the world could have ever let us have it anyway.
There are other measures of 'contributor' that we weed out with our interview processes. Some of those are qualities that working programmers are more likely to have.
For instance, work-a-day developers tend to be less susceptible to hubris. Sometimes when you have a problem to solve, you want the no-bullshit person to do it. It's way easier for me to talk someone out of Imposter Syndrome or at least convince their boss to ignore it than it is for me to dial in the degree of Kruger-Dunning my faster-talking coworkers are covering up.
Luckily actually working in software development is a hell of a lot more like practicing than performing, and there's usually no-one watching you, so I'm fine at that part.
[1] described at https://news.ycombinator.com/item?id=11662380 if you don't recognize the term
Speaking of the second chance, from the link you referenced https://news.ycombinator.com/item?id=11662380
In that comment you said
> I want to turn this system into something that's open to all users who want to take time to review stories. We'll make it a form of community service that will be a new way to earn karma. However, it's still an open question how to pull this off without simply recreating the current upvoting system under another guise.
Have you guys looked or thought about this any more since then? I'd love to see such a community service become reality.
Edit: if anyone wants to be a story reviewer, find 3 submissions that got posted to HN but didn't get attention, which you think the community might like and which gratify intellectual curiosity (i.e. aren't interesting just for sensational reasons). Oh, and which you're not personally connected to. Then email them to us at hn@ycombinator.com.
For examples, you can see a partial list of stories that were picked in this way here: https://news.ycombinator.com/invited. Those are ones where we invited a repost because the original submission was too old to re-up, but the criteria are the same. We'll publish the full list at some point.
The same is true for writers: most are likely not to be interested and/or able to do their best work when watched over the shoulder by another person.
The /real/ question is this: should software be like a factory job? If so, let's clearly acknowledge it and define it, so that people can set their expectations and self-select accordingly.
It has all of it: creative jobs, theoretical jobs, and factory-like jobs. White collar and blue collar jobs at the same time.
What we suck at is admitting which jobs are which types and assigning the labor accordingly (as well as setting targets).
Huh, never thought about it that way. That's a very useful perspective. I will start incorporating it going forward.
Possibly this sort of thing is the majority of the work although not having worked for them I don't really know.
I'd also add that by saying modern factory work is quite different from Henry Ford's day, at least in developed economies. There aren't many factory jobs in the West you can get without a qualification.
At most of my companies, we tried moving exclusively to take-home interview problems for this reason. Short problems that could be solved in 2-4 hours of time, as benchmarked against current employees during daytime hours.
Inevitably, some candidates hated this so vocally that they'd ghost us, or some times even take to social media to lambaste us for trying to take away from their free time. Or we had people refusing to do the toy problem (not real work, same test for all candidates) unless we paid them hundreds of dollars to compensate their time.
It didn't matter how much we tried to explain that the entire purpose of the take-home problem was to grant flexibility to the candidate. A large number of candidates vocally hated any interview technique that didn't involve company employees giving 1:1 interview time to them.
We just filtered those candidates out of the pipeline, but I walked away with a clear understanding that interview practices will never make everyone happy.
Such collaborative interview setup also addresses interviewee concern that interviewer may not value interviewee time.
- only after I had a face to face (can be on skype, but not on phone) interview with a hiring manager/engineer, so I know I really want to work for that company. Job ads usually not very useful determinating that (Interview should work both ways.)
- Not used as a prescreening. It should prove that I really know what I'm talking about. Otherwise, maybe I'm just wasting my time (See above)
- No more than 2-4 hour (we agree on that)
- We review my code. It's good for both of us. I get feedback, the Interview gets more inside why I did what and if I really write the code. (Optional, but prefered)
It's otherwise billable time, don't see why you'd have a hard time respecting that from a candidate.
Sure, as long as the business gets to deduct the cost of the interviewers' time from that. Not gonna work out in favor of the candidate but respect is a two-way street.
After all, once the candidate (or a candidate, anyway) joins the team, it's their billable time being spent reviewing and interviewing other candidates too.
They can at tax time, whereas an employee can't. If an employer requests work from someone, they should pay them.
This is true, but it's not a problem. The problem would be everybody using the same interviewing practices.
Me, I think complaints about spending a few hours on an assignment are ridiculous. All the alternatives are much, much more objectionable to me. The main fallback I have is a contract or temp position, which means my "interview assignment" takes months.
(A: Well, he won't be much longer!)
He writes extremely well. He summarizes a great deal of feedback and knowledge really well. I don't care if he can code -- though evidently he can. Who would pass him up??
Granted, there's the question of whether he fits any particular job opening. But still, there must be many job offerings out there that he could fill if only the people doing the hiring could see that he's got something very special going on.
If you work in a team, in a large company, you need great communications skills almost more than anything else. He's got that down pat.
Jared Nelsen is going places. Ten years from now, when I see his name in the news, I'll enjoy knowing that I saw this a decade earlier.
“If you run into an asshole in the morning, you ran into an asshole. If you run into assholes all day, you’re the asshole.”
This pretty much sums up my impression of these hiring managers that crank the bar sky-high: they are assholes through-and-through, and are full of shit to boot.
If you crank the bar sky-high, all you will get are two kinds of people: those that interview/test well, but fail at the job, or those who can do both but will put a crater a mile wide into your payroll due to their high demand garnering an equally high paycheque.
Anyone who isn’t quite at nosebleed levels of competency can grow into their roles and skill sets. Yes, they won’t be able to hit the ground running, and yes, they will have a higher than comfortable drag-to-thrust ratio, BUT THAT CLEARS UP IN TIME, WITH SUFFICIENT GUIDANCE AND MENTORSHIP.
I know everyone has to start somewhere but from the company’s perspective, hiring a junior developer doesn’t make sense.
When did “not able to solve leetcode-nuclear puzzles while singing and tapdancing backwards”, become “junior developer”?
The OP is talking about “sky-high” levels of (interview) competency, and you immediately jumped to the conclusion that anything less must be juniors who need training to wipe their rears.
I think I see the problem in this field.
I’ve met “junior developers” who couldn’t translate simple business requirements into code.
The stereotypical “developer” who couldn’t do FizzBuzz.
Well, first...you were replying directly to a post that was talking about whitebord/algorithm questions and how they’re out of control. On an article about same. So my answer is: ”because I bothered to read the post to which you were replying.”
Regardless, if you haven’t done an algorithm interview in “over 20 years”, you either haven’t changed jobs, or you’ve had a unicorn career. Either way, you’re clearly not representative of the vast majority of devs. True or exaggerated, your experience is so exceptional that it isn’t worth discussing.
Not everyone’s career has been the r/cscareerquestions “learn leetCode and work for a FAANG”.
These tests are ubiquitous. If you honestly haven’t hit them, you’re a unicorn.
I’ve asked around about my coworkers experience over the years because I had never heard of algorithms style interviews until three years ago when I started reading HN and r/cscareerquestions.
I thought all interviews were soft skill, language trivia, diagramming models and architecture, and explaining past projects.
You dont have to stop the projects when people on them change jobs either.
Are they going to contribute as much as a senior dev? No, but they can still contribute in significant ways. They can still fix bugs and find issues in your system, bring a fresh perspective on design and take ownership over smaller parts of the codebase.
The reason why junior developers jump ship is because companies aren't willing to even promote people. So of course developers (of all levels!) are going to leave after 2-3 years when it's the best way to actually get paid and not taken advantage of by stupid CEOs that would rather have high turnover than a happy workforce.
One even hired McKinsey to analyze the issue. If they had given away the McKinsey fees as raises it would have solved the issues easily.
Turtles all the way down. Once a game shifts into zero-sum, it's not gonna stop getting played until it collapses.
Normally, I would say to avoid entering into any zero-sum games. But the world has made a sharp turn towards zero-sum over the past 5 years, so that advice has rapidly become impractical in today's climate.
Wisdom shall lead again at some point, but I don't clearly see when. Threads like this one though are a promising sign.
Also if you are going to lose employees because you can’t match competitive salaries than i would say you are losing them if all other things are equal.
In another words this greatly oversimplified version of events here only allows for you to come to that conclusion and dismisses how an organization could adopt different practices, values and foster organizational culture that combats these issues differently which I think training is an essential component.
I actually think this is a good argument for licensure like civil engineers etc have to have, because one thing it mandates is required training hours (among other reasons I won’t diverge into here for the sake of staying on topic)
You can debate what that should look like and I encourage that but I see little downside to forcing employers to acknowledge that all employees may need some time just to train on new things and different concepts away from the day to day work. I know this is a little bit of a tangent here but it’s something I see so often at every level of an engineers career path yet organizations so rarely set aside adequate time for it.
Why wouldn’t a developer leave for more money and more learning opportunities?
2. If that’s HR policy and you request for an exception and it’s denied, that may be a sign to start asking more questions.
3. You’re assuming that’s HR policy. Again, example dictates you have one option without actually thinking through how an organization could position themselves differently. Of couse salary is a component and rightly so, but at a certain point it’s just not everything
Think of it this way: lifestyle, ancillary benefits, “perks” etc can be all valid reasons to stay a place that treats you well and displays real loyalty (which is often displayed in more generous long term planning like big 401K contributions and fully paid health insurance for the family, open source software time etc). Can’t say I would just arbitrarily leave that for an extra 20% because that 20% can be easily eaten in other ways. Not to mention transparently knowing that your job won’t evaporate if the economy gets soft is a win. All this salary gloating is fine until that happens. 1999-2001 was not a fun time to be in the upper band taking pay cuts
Let’s not grossly oversimplify this. I’m never going to say don’t maximize your value, you should, I’m saying there is more than one (salary) way of doing that.
FAANG or not.
Oh and one dirty secret FAANGS don’t tell you? They happily underpay on salary once you’re in, or freeze you into a specific department etc. it’s not all roses.
I’m nowhere near west coast salaries but just big city America salary for context. At my age now in my mid 40s, married, making about the average salary of a top IC in my market, with the big house in the burbs (again not bragging, any developer with 5-10 years of experience could easily afford it), with the white picket fence and 2.1 kids, yeah 20K more wouldn’t make a difference in my lifestyle.
But, when I was just starting out in 1996 making $33K a year, $20K meant a lot.
But salary compression and inversion are real phenomena where HR policies don’t allow for raises to keep up with the market.
I was working during both 1999-2001 and 2008-2011. In 2000, it was very much just a dot com thing. If you worked in corporate America as a standard Enterprise Developer you weren’t really affected.
2008 was rough, but still there was no such thing as job stability. Companies were laying off left and right. I survived three rounds of layoffs - barely. But even then you could find contract gigs if you were the perfect fit.
That is to say I got extremely lucky. I know that. My takeaway since (I’ve been a software engineer for ~10 years now) is that the way I was treated as a junior is what I described throughout this post, and my mentors were the ones that drilled home what the hidden costs were taking jobs based on salary, how to properly evaluate benefits etc. I learned a boat load.
So I want to preach to developers and companies alike that things can and should be evaluated differently and to avoid pitfalls that are easy in this industry
I’ve worked at smaller companies by choice or at smaller divisions of large companies (occasionally). One developer salary is a major cost and can add to their revenue.
If you only have a few openings, you better make them count.
For instance, I know that some features I designed and developed from the ground up was the difference between us getting a client (B2B) and not getting a client. I also know how much revenue that client added directly to our bottom line. I was what is usually called a Single Responsible Individual. When one hire needs to be that impactful, there is no room for training juniors. All of our local software engineers can point to a feature they spearheaded that added measurably to our revenue.
Now the hard truth is, why hire junior developers locally when you can hire people with much more experience overseas for the same price? I know the stereotype of the crappy outsourced developer, but I haven’t found that to be the case.
Anecdotally, I was lucky enough never to be a junior developer[1], I got my first job out of college with a small company based on an internship the previous year. My first assignment was create a networked data entry system in C that was the basis of a new department that they were starting that allowed them to double in size. But I had been a hobbyist programmer for a decade. By the time I got my next job, I was already considered a mid level developer.
[1] yeah that did hurt me later on. I didn’t get any type of real mentoring and stayed an “expert beginner” for a decade.
For what it’s worth I work at a small company now that is a startup and this is how we do things, to great success, the way I described it
To be clear, I do not work in Silicon Valley either, never lived in SV. So I get that part of it
I’ve seen the argument about paying them market rates. But honestly, if they are young and unencumbered, their “market” is the entire US. Meaning you may not have been in the position to pay them what they could make if they were willing to move across the country.
To go into more anecdotal detail, I had been programming as a hobbyist since I was 12 in assembly for 10 years by the time I graduated from a no name college in the south. I had two job opportunities - one making an entry level developer salary writing COBOL [1] in a slightly larger city, or working as a computer operator in a major city. I chose the latter job that paid much less just to get to the city. I got lucky and I was the only one who knew how to program so they gave me the previously mentioned project straight out of college. They gave me a raise the next year to that of an entry level developer (I was hourly at first and got a lot of overtime).
But now, three years later, the project was done and with a major green field C project under my belt, my market value had gone up more than 50%. They couldn’t justify paying me that and I got another job.
Now if a similar scenario had played out in today’s market instead of 1999, even if they could have matched my local salary of $80-$100K for a mid level developer, I could have spent a year studying “algorithms and leetCode”, picked up and moved to SV and doubled or tripled my total comp. There is no way they could compete with that.
Let’s look at the current state of affairs. Almost every engineer at my company is between 35-50. They purposefully created the office in the burbs three years ago to attract older developers. They not only get experience, paying local market salary is relatively easy comparatively. The chance of any of us uprooting our families, selling our big, relatively cheap 3000 square foot houses in the burbs with the great school systems and moving to the west coast is slim.
We train developers in popular languages (particularly our stack leverages Vue and C# and a good deal of custom SCSS) we are only ~3 hrs from the valley yet we aren’t losing engineers in droves and haven’t ever for the years we have been in business
You should pay juniors market rates for their positions, period. I don’t agree with that at all
https://www.businessinsider.com/average-employee-tenure-rete...
This logic is sound, but I don't personally agree that bureaucracy is gonna solve any of our problems.
The rest of your comment, I believe if you spend time reading my OP carefully and spend some time noodling over it then you'd find that all of it had been covered already.
I don’t buy gate keeping either as the number one issue, see the lawyer glut as an example.
That is a zero-sum attitude that does not hold up when a company does other things well. If you are training your employees, you are likely to be doing the other things that help retain employees (https://www.bayshorestaffing.com/images/uploads/Blog_Image.j...). And if you are focusing on and actually delivering all ten of those benefits, employees are extremely unlikely to be jumping ship unless they are either a poor employee or have poor character (which you don’t want anyhow), or the competition is paying not only well above market rate, but well above your rates. Which means that they are now at a strategic disadvantage because an inordinate amount of funding needs to be diverted to payroll - much more so than in a company like yours.
Any company can make an end run around zero-sum thinking by fully and honestly employing those ten benefits. Employees can and will rise to the occasion, bringing a competitive advantage to your company that is well in excess to their numbers.
Why? The only guidance then a more senior engineer would need to give is during code review and where they may have questions. It’s a great way to get them familiar with the code base and train them up. I haven’t seen this be a productivity killer.
Also, with the shocking frequency I’ve seen, code based need more tests. Have them write tests! That’s a fast way to get up to speed.
One other point: senior engineers in one code base may have an uptick if a “junior” in another depending on existing quality, circumstances etc.
I think organizations are lying to themselves that they can only hire seniors. It’s a myth. Just like 10x developers.
I need hard data to be convinced otherwise not anecdotal evidence. All the hard data I’ve seen points in the opposite direction of any anecdotal information I’ve come across.
There are so many volumes of books devoted to these topics alone that I think most ideas around developer productivity without the firm backing of numbers are mostly hand wavy and myths we tell our selves to feel better
We had juniors for easy new features, tests, anything that is small and isolated.
These are just some things I’ve been apart of. I’ve gladly been apart of this before.
You can as I said also have them write tests or as others noted have them write smaller features or other things in isolation.
Also how green are we talking is another question entirely. Some tasks are better fit for people with maybe practical experience in another tech stack but need to get up to speed in a new language etc.
My overall point which may not have been clear is that there is always a way to get an engineer up to speed without killing velocity. It’s a management failure to say you can’t hire junior engineers.
It’s one thing to need someone that’s a domain expert, but often I see “senior engineer” === “No ramp time” and I think that’s just a lie management tells themselves to justify not coming up with a good on-ramp plan for new engineers
Functional tests are a must. I would also say that in this day and age integration tests are also a must. I don’t think there is any domain now where you are commonly interfacing with only your code end to end, some level of integration is happening and that should be tested too.
Nothing’s perfect because we are human, I will say though that having proper functional, unit, and integration tests for a codebase I don’t see this issue crop often and when it does it becomes apparent, and then you write a test for that and then you have coverage for that etc.
Software is an organism composed of changing structures, after all.
And one of he best ways to make sure your senior people always have something new and interesting to tackle is having them mentor junior people to take over the things seniors are bored with.
Corporate America is infamous for HR policies where they will only give current employees a maximum percentage raise while hiring new people at market rates (salary compression and inversion). Even if you as a manager want to keep good developers who were juniors but have now leveled up, your hands are tied.
However, most of the people I know who start looking for another job don't do it because they suspect there's more money somewhere else. They do it because they've grown less happy with the job.
So, it makes sense somewhat.
I've had recruiters talk to me at length about each of the people I'd be meeting in the on-site, their backgrounds, what they'd be covering. I've had ones give me broad strokes about why previous candidates were passed on. At the end of the day, it's not in their interest to bring too many people to an on-site who will fail spectacularly, because that means they're not doing their job properly.
I may just be lucky.
1) You've had the good fortune to work with good recruiters/HR reps, and
2) You're a qualified candidate for the role they're bringing you in for.
I say that because of an HN comment I saw from a recruiter (maybe Aline Lerner? Don't remember) that they typically send in one unqualified candidate to interview for a role first, no matter what. That's because the hiring committee always - always - has an itchy trigger finger to reject the first candidate. I'd have to imagine that a recruiter worth their salt would learn this lesson as well; you don't get good at something like recruiting without learning similar hard lessons as the other good people in the field.
Point being: the recruiter was trying to give you an idea of what to expect in your on-site, because they thought you were good, and they thought you'd be introduced at a point in the process where you could actually succeed. (The hiring committee having already gotten their itch to reject a candidate out of their collective system.)
Turned out he did 500 interviews a year. Five hundred. Unless that's your entire job, I don't see how you could come out of that without becoming an asshole.
From my own experience, that sort of load undeniably turns evaluators into assholes. I think the main problem is that to see any return on investment (i.e. a 'successful' interview) it takes about a 6-12 months assuming the person is hired. Having to do such a repetitive task with such a dearth of feedback I think will make anybody go crazy. That's not even considering any imbalance between time spent on recruiting versus time spent on strategy or other management tasks.
Most jobs are not like that, and then you get extreme variance in expectation with little communication on how to prepare, which is a waste of time for both the interviewer and interviewee if the interviewee has no clue what will be asked or expected ahead of time.
It would be way more time intensive for the interviewee to be asked to code up some fullstack project for every interview.
Having to memorize some basic algorithm questions that are maybe 20 lines of code each is way better.
Assuming the role is a fullstack developer, memorizing basic algorithms doesn’t show one is fit for the purpose.
Providing a simple skeleton of a fullstack project—in the chosen tech stack of either the candidate or the company—and then verifying a candidate can add a simple feature, or something similarly fullstack, would accomplish that far faster than algorithm answers.
Edit: I realize this risks sounding like stupid take-home interview homework. I personally oppose that crap. However, I recognize why some companies take that route, as I don’t think I’d feel confident that a candidate could work in my company’s stack by asking silly algorithm questions. I’d probably feel more confident watching the candidate do a remote screen share, git clone a starter app, and do some simple to moderately complex fullstack tasks. Of course, the tasks should fit the role, I think—e.g., I wouldn’t ask a candidate who’s being hired to tune DB queries a bunch of fullstack questions. And if I was hiring a backend dev to build out APIs, I wouldn’t bother with a bunch of frontend tasks and questions. The hiring processes I’ve seen and managed always had better results when more time was invested in prepping specific, job-focused interview processes, rather than offloading that time onto candidates because recruiting teams can’t actually do more than ask shallow questions or follow checklists.
Yes I agree but the current way we have is not even close to that.
He touch that issue in the article. So you spend a lot of time and effort to learn about binary tree, great, you pass with flying colors with company A, then you interview with company B, they ask you trie tree ...fuck.
If the interview process is largely memorizing 200 or so commonly asked algorithm questions and that is the gateway to a $200k+ job then it's a good thing for applicants, not "dystopian" at all.
Again, it would be much, much more painful and time consuming for the interviewee if they were asked to code up some fullstack project for every interview. That is far more time consuming, and in my opinion more "dystopian" to expect those interviewing to do dozens of hours of work specific to one interview for free.
Part 2, paying someone on trivia instead of capabilities is not sustainable. The company ends up suffering in the long term. Other engineers that are actually capable have to pick up the slack. Longer hours, less family time, higher burn out risk. Then comes the firing period because the company is losing revenue due to rampant incompetence. Putting even more pressure on the capable engineers. There's plenty of articles where trendy startups have some brutal layoffs, even though 12-18 months earlier had massive funding rounds and went into "extreme" hiring phases. The chickens come home to roost, no matter the sparkling bling of big paychecks.
For most of us outside the Bay Area and NYC, it's quite unlikely that we'll secure a $200k/year job unless we go into management.
*I think Austin is the smallest mentioned.
At 284, Boulder, CO is considered a "city". Don't get me wrong, it's a nice town, but you can't compare it to Austin (11), Portland, OR (25) or NYC (1). Top 100 cities comes to ~64.5m (Spokane, WA coming in at 100 with 219k).
I just... wow... I don't know why, but I truly thought "city slicker" America made up like 50%+ of the population. At best it's 29%. Not insignificant. But... not a wide majority.
Then take Wilsonville, OR. It's oddly considered a metro area for Portland. It's a good 30min away and independent AF from Portland. Plus the people there are different. More old money or straight up white trash.
Metro is a really loose/gray term. To say surrounding towns are the exact same as a city is a bad idea. Let's take the purchasing decision of a home in those areas. The "value" of location compared to price and are at odds. One person is willing to pay a premium for location, the other is not. Those are two different mindsets as to what that person is willing to deal with in life.
And the overall lifestyle of a place depends on the population within a reasonable radius of travel, not what is within an arbitrary boundary.
I live in an area of NY currently that has towns similar in size to where I went to school in AZ, but the surrounding area, say a ten mile radius, has maybe three times the population. It's a very noticeable difference, both in general lifestyle and in employment opportunities, education and so on.
Yes. Maybe you can understand some of the frustration coming out of "middle America"? The new American economy is booming in large metro areas, while the rest of the country stagnates. I'm not a Trump voter, but I certainly understand the bitterness at being economically and culturally dominated by a segment of the country that is at the very most, 50% of the population.
If you know plenty of people outside of the major tech big cities (and I should have included Austin and Seattle in that, for sure) making over $200,000 year, they you have a statistically unusual sample of friends.
All the data back up what I'm saying. Go look at the numbers. Look at salaries on monster.com or your favorite job site. Perhaps you and your friends don't realize how unusual it is (granted, you also have higher cost-of-living in those cities, sometimes by enough to eat up the additional salary).
Maybe I should revise my claim to "top software engineers make over $200,000 in Austin, Pittsburgh, Seattle, etc.".
Not to mention such interview styles can be gamed. Suppose a Flatiron bootcamp for DS&A questions becomes big in response. What then? An arms race for more and more difficult weeder questions?
Such questions aren’t necessarily bad, but focusing on them to the exclusion of all other skills is becoming an anti-pattern.
Maybe but its not the purpose to select people with families or commitment. You choose to have families or commitment, you have to deal with the trade off.
>An arms race for more and more difficult weeder questions?
Its always be an arm race. Why to expect otherwise ?
The people who do the hiring get to the decide what the metrics is and what they considered good/qualified.
Yes, that what said, but I doubt according to the people who do the hiring.
It doesn't matter if you think you are right candidate according to you or other people. Ultimately its the people who going to hire you who is going to judge you according to his/her subjective criteria.
Hiring is still an on-going debate. Google SVP of People Ops Laszlo Bock has admitted previously that internal studies revealed that the interviewing practices at the time didn't really correlate to employee success:
https://www.nytimes.com/2013/06/20/business/in-head-hunting-...
https://news.ycombinator.com/item?id=17196974
We're having a debate here. You can claim the current practices are best, that's fine. But you can't just state it and take it for granted without providing evidence.
> Four meticulously orchestrated Google interviews could identify successful hires with 86 percent confidence, and nobody at the company—no matter how long they had been at the company or how many candidates they had interviewed—could do any better than the aggregated wisdom of four interviewers.
So it isn't that interviews have zero correlation, it is that they fond no case of a single interviewer being better than the aggregate of four algorithm interviews. And as you see, four algorithm interviews combined have a very high success rate.
https://www.theatlantic.com/business/archive/2016/04/the-sci...
Families are kind of important. They're usually a large part of the reason people have a job to begin with. To callously disregard the impact of a system on families is exceptionally appalling. You should not do that.
This isn't unlike other professions, but you would lose self-taught developers who don't have the time/money to afford the credential. That is currently something special about development compared to many other professions.
This assuming you prevent people who can't do something like fizzbuzz from graduating with a CS degree.
1. False positives: you get folks who do really well at DS&A yet who are really bad developers. I mean really bad. I wouldn't have believed it if I had not seen their interviews and then subsequent performance. I'd wildly guess that it's about 20-30%.
2. False negatives: you get folks who are really good developers, and yet for whatever reason, perform badly on DS&A algorithms despite practicing. I think this number is higher than false positives, probably around 50% or more.
If you're a FAANG company, you can afford to play these odds. If you're not FAANG, then you're killing yourself by requiring DS&A interviews, almost inevitably.
Both are contributing to destroying the profession for huge numbers of people, IMHO. I mean, that's good for me, because I'm almost certainly sticking around and generally do OK on DS&A interviews with adequate practice (which is a waste of time, since anything beyond a broad knowledge of the performance characteristics of various DS&As is totally unneeded for 99% of us). But it's not right, and I really don't like it.
However, I am in consulting, and was on the bench. My manager forced me to do an interview for coding. The questions were absurd. Literally reading vocabulary terms and looking for exact definitions to things.
I continually asked what their problem was, what they were trying to solve, what steps they faced, etc. They just kept asking questions like above. And every time I'd get it "wrong" they'd say "we were looking for... <googled definition here>.
Honestly what is the point? I feel really bad for kids today. I get all my jobs through word of mouth and networking now. Why? Because it's much easier and those ways almost always have better outcomes.
Strange postscript that suggests you hold some problematic opinions that indicate you can't play nicely with others?
Did you fail to get the job because you didn't have the tech chops or is it because you said something wildly inappropriate on the behavioural questions?
What did you say?
It wasn't "triggering" or anything! I happy gave my opinion and moved on to the other questions.
"Are you a leftist? We only hire leftists."
So if you don't want to hire assholes who think "diversity" is a dog whistle, you hire leftists.
Inclusivity does not mean having to include those that would exclude others.
But if you exclude all non leftists, that's about 2/3 of all Americans. And that looks very tribal to the rest of us.
The mandatory brilliant essay on this: https://slatestarcodex.com/2014/09/30/i-can-tolerate-anythin...
But that's being overly provocative.
Ultimately this started with you saying that a question about diversity could be heard by some as a dog whistle that only leftists are welcome.
I simply don't agree with that premise. I do think there is a percentage of people who think diversity is a left-only issue, but I don't think that's 2/3rd of Americans. Then again, Donald Trump hovers around 50% approval rating regardless of what he does, so maybe I have too high an opinion of Americans.
I work at FAANG as a backend engineer, and so I don't know this one either.
This is almost certainly to the detriment of employers.
I think if employers understood the true cost of their actions, they might radically change the interview process. Everyone wants the best talent, until you consider that your talent is actively studying to become someone else's "best talent".
In my opinion, and modest experience (I've done several dozen interviews, so there are plenty more experienced then me, but I'm at least not new, plus nearly all interviews were for my team so I had to live with the results!) it isn't that hard to come up with small sample questions related to what the job actually is doing. From there, in addition to the mere fact of whether or not they were able to complete the task, you can learn about how they did it, look at their practices, stop them for a moment and discuss the implications of something they just did. You can add a wrinkle to the task ("what about unicode?" "what about cross-site scripting vulnerabilities?"). I also find being this kind of flexible, rather than being stuck on "did they solve this algorithm challenge in the proper manner" also gives me room to be human, to take a moment to try to calm the candidate down in what is inevitably a stressful situation for them (even if it's basically just Tuesday afternoon for me in my current position in the world). I think of an interview as "I want to find all the positive I can", rather than as a process of locating all flaws, and you can't do that if your candidate is frozen in the headlights the whole time.
Interviewees are pouring forth a wealth of information. The idea that the only way to find out how good they are is by asking this one narrow set of questions is absurd.
And the interviewers are pouring forth a wealth of information too. Consider what message you're sending with a rigid adherence to something we all know is broken.
[1]: Unless it really is relevant to the job! I expect that if I'm hiring a senior level machine learning researcher that you better be able to describe gradient descent to me, for instance. We may not implement it literally in the interview but I have a reasonably case for saying the researcher ought to convince me they could get there. But in a lot of cases, what's more relevant to me is that you know the characteristics of algorithms rather than exactly what they are. You can know the runtime and weaknesses of quicksort or RAFT consensus even if you can't spew the actual algorithms on to a board.
Actually, my parenthetical here almost makes me want to rewrite this whole post, although this thought is still fresh and I'm still chewing on it. Is that perhaps the error we are making? Are we conflating knowing the algorithms for knowing the characteristics of the algorithms? The latter is legitimately important and I see screwups based on failing to understand characteristics of algorithms all the time. Knowing the literal, actual algorithm to the point that you could simply open up a terminal and bash it out yourself is rarely important in this era.
I left Google in 2009, quite unhappy about the difference between a startup and corporate politics. After that I worked at 3 startups (5 years), 1 telco (1 year) and at McKinsey (3 years).
I ended up interviewing at Google again in 2018, after deciding that a) I liked programming more than management consulting and b) my comparative advantage was in programming. I definitely could have felt entitled after 1) passing Google interviews once before, 2) having had three somewhat successful startups and 3) proving I can also handle the corporate side.
I was asked multiple questions that could be interpreted as 'algorithmic' (and I have to admit that after 3 years of not coding I was nervous about them). However, they can also be interpreted differently, more charitably. They all followed the structure of: 1) give a reasonable but not completely defined problem - you should be able to clarify enough of this to show you can deal with some ambiguity; 2) turn that into an algorithm - you should be able to articulate an easy algorithm and be able to discuss improvements to it; 3) turn your algorithm into code - you need to show you can actually program. I definitely didn't have to memorize and regurgitate textbook solutions.
I acknowledge that a) the above may optimize for rejecting all negatives at the cost of some false positives and b) it's still a very basic lower bar and doesn't predict who will perform highly vs. ok. However, I did think it tested for the core skills I expect from programmers.
I've also set up interviewing processes at startups. I did end up emphasizing ownership and 'getting things done' more than Google does. I do think startups should test for different attributes than 100k+ FTE corporations.
If someone just spits out an optimal solution straight away that's not actually a great sign. It doesn't matter if you struggle a little, or need a few hints, but showing you know how to break a problem down into smaller sub-problems and work your way up is an invaluable signal.
I think I tend to make good impression on these ... but I am aware it is interview skill where I somehow picked up social signals I am supposed to send. I am adjusting what I am saying to how interviewer looks like (happy, annoyed, bored, etc). It is social skill, but not same social skill as pretty much anything I do in actual work. In actual work, people are not interested to listen to me work out problems in front of them.
It is even more useless if you know the answer and then proceed pretend to split it into smaller problems and then work your way up or down. That neat thought process is neat precisely because I know the answer and I am performing idealized problem solving process.
In case of actual real problem, you don't do that so nicely. You have at least some bad turns, random guesses, break it multiple times badly and so on.
Yeah. In a world where Google exists (or even where Knuth exists), what's the value in having algorithms memorized? Knowing the characteristics, sure. I agree that that's important. But the algorithms themselves? Why? I've got better things to do with the memory space.
"Gluing lots of things together" is a descriptive term, not a derogatory one. There's a ton of things you need to know to be a good gluer nowadays; security, performance characteristics of all the pieces, we're adding more and more distribution into our systems, higher level stuff like integrating with teams, documentation of your code, how your workflow goes... there's more than enough to interview all day on these issues without ever having to quiz the candidate on puzzles from the back of a Knuth book.
(This is agreement with you, not disagreement.)
I think this really explains much of the mystery of the 20 years of experience engineer who can't pass fizzbuzz. Writing code on paper/whiteboard is completely different from a text editor.
Just think about how you write code. You write a few lines. Run it in your head. Notice a mistake, add a newline, remove some brackets and it's all good. Oh wait, you just noticed this can be simplified. Ctrl-A delete all, reduce entire 20 line function into 5 lines.
None of this is possible on paper/whiteboard, especially in a timed interview setting. Paper/whiteboard is effectively immutable. You get one chance to write it down correctly and that's it. There's just no time and it's too difficult to even add a newline somewhere.
So you put a 20 year programming veteran in front of the whiteboard and they can't do fizzbuzz because writing code like that is so foreign to what they do. They can't just write something and fine tune it, which is how software is written. It must be perfect on the first try. they freeze up and fail.
There is zero excuse for companies to not give a laptop for interview coding. Even if it's Linux boot CD with nothing but vim and nano, that's still so much better than whiteboards. And these days companies use hackerrank during the phone screen. Just use the same website on a chromebook during the onsite!
Writing code on whiteboard is just really stupid. Can we all agree on this at least?
When hiring 40+ people for a neo-bank, I did what you should have experienced: listen to people and get to know them.
The key point of success in a team is of course their skill set, but also the potential synergy.
Lots of developers are very bad if not terrible at presenting themselves, be it on a résumé or during an interview. And this has no relation to their actual skills.
So I always try as hard as I can to actually extract information from them.
First, I let them feel at ease, telling them that this is not a challenge but a time to get to know each other (which is actually true).
Then I ask a lot of questions about their experience, both in the technical sense, but also human one.
I try as much as possible to understand how they think and feel, what do they value, aspire to, try to achieve by joining the company.
I’m not a hiring manager but a 15+ yrs of experience developer. Yet, I find that hiring developers by actually being one goes much smoother than when you’re not.
Last but not least, I think that this crazed interviews are becoming more and more common practice because developers agree to pass them.
If you want to change this, then don’t go through them. There are thousands of other great companies that do no torture their candidates.
I thought I was defending it: https://news.ycombinator.com/item?id=22332336
1) Terrible Algorithmic interviews, AI-based evaluations, whiteboard coding.
2) Hey you are hired! Now move to $BigCity, with expensive real estate, schools, transit and everything else.
3) So, welcome to the job. You'll be maintaining this shitty old python 2.4 CRUD app that multiple people have tried and failed to upgrade. And one of the web guys wrote his own UI framework for it, kinda based on Backbone but with a lot of custom stuff. Yeah he doesn't work here anymore. Anyway that part is in coffeescript, no we can't change that, too expensive to rewrite.
4) But don't mind that, because you'll spend half of your time in Agile meetings! How are you with story point estimation?
5) Hmm also we're behind on the schedule. You don't have a problem with 10-12 hour days right? And no we can't cut back on the Agile meetings, that would hurt productivity.
6) Open floor plan! We think this will help productivity a lot. But sorry, no noise cancelling headphones, that will hurt interactions. Also the VCs love it when they tour. Nothing like butts in seats.
7) Here are some options! But they are probably worthless, because we'll never be successful. Even if we are, we probably won't go public. Even if we go public, your share is probably worthless because of preferred shares and liquidation preference. Oh yeah, there is a 4 year vesting schedule and its ridiculously expensive to buy the vested shares if you leave the company, which is of course intentional since we don't want people doing that. Anyway maybe you'll make enough money for a new car, eventually. Imagine commuting in some sweet new wheels, 5 years from now!
8) Sorry, only a 1% raise this year. Health insurance costs and all that. But if you like, goto step 1 with a new employer to try to get a real raise.
9) Yes we are aware that the expense ratios on the 401K are 2%, and you are only allowed to invest in target retirement 20(fn($YourAge)). Indeed that sucks, for you.
10) By the way you only have maybe 10-15 years of useful programming life before age discrimination kicks in. So you better get crackin'
But I'm concerned that the popular perception is: go to a coding bootcamp for a few months, or even study CS at university, and that is your golden ticket to a solid, happy, lifelong career. Which I believe will be untrue for many people.
Yea, if I had also been working on this particular code for the past two years I'd be able to breeze through these problems too.
One person's "ladder pulling" is another person's "raising the bar".
Over time, companies should continue to raise the bar for both candidates and existing employees. As your company grows, you can and should become better at attracting more talented and ambitious employees. You tend to have more resources to compensate new hires better, allowing for more selective hiring practices. Growth means you've learned how to more effectively identify the best candidates.
Companies aren't being vindictive or selfish. If they can't hire enough people, they'll dwindle and die. The bar should be set just low enough to keep enough talent coming in the door, but no lower.
Companies aren't obligated to hire talent at the same levels they did in the past. It's not ladder pulling, it's just business.
That's largely due to the position taken up in "The dystopian world of software engineering interviews". Had the article been a defense of current interview practices, you would have received sentiments to the opposite effect.
Here's a comment on the original thread doing what you said "not a single person" has done: https://news.ycombinator.com/item?id=22332023
I agree that this was a cheap shot from the author.
The current zeitgeist of hiring practices is that we're all obligated to chant "interviews are broken" without ever suggesting alternatives. Suggesting alternatives is dangerous because someone, somewhere will come up with a reason why your suggested practice is bad. Even the author of the article avoids suggesting any alternatives.
It's the equivalent of politicians saying that they're "deeply troubled" by something, and then proceeding to do nothing because they are secretly okay with the status quo.
No one is dumb enough to go out of their way to publicly argue in favor of the current hiring practices in this atmosphere. Yet when we're put in the hot seat on a hiring committee, we all fall back on the tried and true methods of interviewing for a reason.
Nah, there's somebody who hates everything, but there are plenty of things that somebody likes. I'd be happy with both take-home problems at a moderate effort level, and Fermi problems. Clearly many people go out of their way to declaim how much they hate those things, and I don't know where to find companies that hire based on them and pay a lot, but it would make me happy.
I wonder if then at some point we see computer engineering becoming more standardized like the others, with a bar that someone must meet before they are able to call themselves a computer engineer with all of the responsibility that entails. I wouldn't say that it should restrict someone from programming professionally without this accreditation, but rather it would be a symbol of proof that someone has at least been judged by their peers to be at a given level of skill and knowledge. What this would look like ultimately or whether it would come to pass, I haven't the foggiest idea though. Like I said, I'm just a QA guy, not an SE.
No doctor is expected to share money with his teachers nor is expected to teach sons of his teachers. Abortion is allowed and lately euthanasia too. Current doctors can perform operations and are not required to treat all sick people.
Haha, that reminds me of the old days of Slashdot, immortalized is this early 00's webcomic:
https://www.pyrocam.com/files/images/clanbob.dundundun.dontt...
There are a lot of anecdotes like that, but the statistics I've seen has clear correlation between performance on algorithm interviews and performance on the job. I don't think there are any public studies done on this though so if you don't work at a company where you can view it internally you just have to trust that someone has gone through the numbers at these big data-driven companies.
I expect, though, that the issue is that the degree of correlation measured will depend on the population you're screening with it. If you are wanting to screen out fraudulent or otherwise utterly non-programmer persons, it will show a good correlation. However, among those who are actually programmers, it will show much less correlation, and if you're in a condition of developer scarcity, the false negatives could be more costly than the false positives.
Do you even understand what correlation means? There is a significant random element to it, yes, but that doesn't mean that there is no correlation, and the random parts from the same individual can be mitigated by doing many interviews after which you have reasonable correlation. Most interview styles has close to zero correlation, so even if we don't reach anything remotely close to 1 it is still pretty good.
Edit: About the points that it is enough to weed out people using simple problems, that isn't true either. There was significant correlation between job performance and interview performance even among those who did well. The statistics showed no signs of it capping out either, so for all we know we could make them even harder, just that there wouldn't be that many left that could pass them then.
I can only claim to have personal anecdotes, but the critiques I hear and have are things like asking you to write Bash on a whiteboard an focusing on minor syntax errors (Bash was not the primary language for the job and the candidate was obviously familiar with the language). Or asking about details from compiled languages when the job exclusively dealt an interpreted language. I'm not saying these interviews shouldn't touch on these topics. I do like to see how deep a candidates knowledge goes (sometimes it can mean they're over qualified and would be bored).
I'm not sure if it's interviewers showing off, because they've "always done it this way," or what, but I often see both job posting criteria and interviews focus too much on things well beyond what the job requires.
Of course, I'm biased and think my way is better. Outside of the minimum requirements for the job I never really care if a candidate has an answer to most questions. In fact, when they get it too quickly it seems rehearsed and I don't think it reflects their skill. For me, the technical questions are a starting point to see them talk through how they approach the problem.
Shameless nonself-promotion: The Discord "Credentials + Meritocracy"[2]
I have masters in two STEM degrees, which most employers seem to value at exactly nothing. They are very enthusiastic about hiring me. At the same rate they would hire someone without a degree, that is.
The thing they want, is multiple years of professional experience with a very specific set of 4-8 technologies. These are 'must-haves', and then there are probably 4-8 'nice-to-haves', which are even more exotic demands. And then companies complain 'that there are sooo little people skilled people in IT' and 'the IT people have such ridiculous demands!'. In my environment, I see people with a lot less experience make a lot more in other fields.
(This is how it goes in the Netherlands, at least.)
About 80% of them actually, roughly estimated
As it had been pointed out here by most people interviewing sucks. It sucked in every country for me (some more than others). The process was superficial and usually didn't give the employer a decent picture about my skillsets nor did I get a decent picture of the company's environment/tech/culture, meaning a "we want to implement a new data analysis platform and ML on top of our web app" was only a nice attraction statement.
Both parties tend to tell some fairy tales what they think the opposite would like to hear just not to miss out on a possible opportunity. It's a bad habbit that I've done myself in the past because I don't have employees knocking on my door every day with a contract in their hands.
One of the issues is probably that a company doesn't know the person who is being interviewed. Just because someone got hired by 5 companies before and has 10y experience doesn't mean they know their stuff and therefore everyone has to go through the same tidious process again and again.
Couldn't there be an evaluation of someones problem solving skills and ability to built systems, that has a value for all (or certain years) of upcoming interviews? Lets say I take a test, that evalutates my "can solve problems" and "can built awesome cloud tech" skills and provides a certificate for it which is valid for 5y. Of course you'd also able to update your skills whenever you've learned something new!
So every company looking for a new employee searches in that pool of professionals with pre-evaluated soft and hard skills for a suitable match. Not like LinkedIn where you can claim whatever you like but rather evaluated by other professionals who will vouch for you.
These companies know in the last 10-20 years who interviewed who and the performance of everyone after hiring. With enough data you can:
- A/B test which interview questions better predict future performance
- how much each interviewer's feedback is correlated with future performance of the interviewee. Some interviewer's are likely terrible and they should not interview. Some interviewers at good at predicting future performance and they should interview much more if not full time.
- which interviewer provides biased feedback (women, minorities, etc)
Either these companies are not analysing this data and the interview practices may indeed be garbage. Or FAANG companies are analysing this data and otpimizing their interview practices accordingly and the process/results are actually as fair and good as it can be.
> Four meticulously orchestrated Google interviews could identify successful hires with 86 percent confidence, and nobody at the company—no matter how long they had been at the company or how many candidates they had interviewed—could do any better than the aggregated wisdom of four interviewers.
https://www.theatlantic.com/business/archive/2016/04/the-sci...
At the same time, a great deal in this is the level of expectations about the "greats", finding the right job place at once. I believe adjusting these criteria might open up a lot more chances to get to doing the craft, be on the team, well, interview the others too.
Good luck!
I thought there was a shortage of software engineers right now? Although I just got an email from Glassdoor with estimated salary (in the Bay Area) being 5-10k lower than the last email, so who knows.
All of this is well orchestrated by large tech companies in an effort to drive down labor costs. The problem is, there's an assumption that the current labor market is just asking too much.
I think looking at the current labor market landscape, practices in place industry wide (e.g., agile, interview processes, etc.) were starting to see labor market feedback that this simply isn't the case. Development is hard, it's time consuming, expensive, and requires a lot of active practice--no matter how much you want to drive labor costs down.
Ultimately I hope the entire strategy backfires and we turn away talent industry wide to other sectors and then there truly are shortages resulting in businesses pushing these practices unable of maintaining their businesses and forcing them to scale back--wishful thinking though.
So at the end of the day you can build more complex software with lesser amount of developers but there is a catch, and that is developers now need to be much more talented.
Right now a good developer can easily develop what a team of 10 average developers could build 10 years ago. This is especially true in GameDev field. Right now there is a silent revolution going on with procedural art-work so that a good programmer/artist can create huge amount of content in very short-time but procedural art generation is really complicated.
So if you are a talented developer you can compete with mid-sized companies right now!
I believe it will get to a point in future that even a small team of talented developers will be able to out-compete companies with 100s of average developer.
Large companies will have no option but to scale back and change their HR practices but large companies, like large ships, are harder to turn, hence some of them will definitely go bankrupt in near future.
Which, in turn, drives the number of bugs and flaws way, way up, which drastically increases the number of developer hours and money spent trying to fix products that are constantly broken. This situation just gets worse over time because those "better" libraries/tools/knowledge-base make adding new bugs much, much faster than fixing them. Evernote is the best example of this I have ever seen.
> So if you are a talented developer you can compete with mid-sized companies right now!
If you're a talented developer, you can compete with what mid-sized companies were doing 10 years ago and large companies were doing 20 years ago. That's more than good enough to make a niche game that pays well, but it's still an order of magnitude or more off from what mid-sized companies make today, both in quality and revenue.
> Large companies will have no option but to scale back and change their HR practices but large companies, like large ships, are harder to turn, hence some of them will definitely go bankrupt in near future.
Unless they're being propped up. Twitter didn't turn a profit until 2018. They spent 12 years burning cash. Most companies would have gone under after their investors pulled their money when they hit year 5 without making a dime.
Perhaps you should stop flocking to one location in the states and go somewhere else?
If there was, hiring and compensation practices would reflect it like in any other field.
It was a two-part interview, each part lasting around one hour. First hour I just talked with a single senior developer. About previous experiences, education, interests, etc.
Second part was a “technical” challenge. I was given a loose specification of a simple fullstack web application, and then asked to produce wireframes for the frontend, an architecture drawing for the entire system, a database schema and a project plan with estimates.
There were no detailed technical questions or exercises. My answers were nowhere near perfect or realistic - but that was also not the point. The point was, they wanted to see my thought process when approaching a real software project.
Why we engineers whine so much about it?
I'm an interviewer at a FAANG company with >500 interviews in the last 5 years, I teach multiple internal courses on interviewing, and I think the system makes a lot of sense for companies at our scale.
Ask Me Anything :)
I currently have a toddler on my lap, but I'll edit this post later today to address some common criticisms and misunderstandings, but also point out which concerns I think are valid and where we should do better.
edit:
Okay, let's go through the most important points
1) I'm a Software Developer. I interview other SDEs. I ask them problem solving coding questions, algorithmic coding questions, data structure questions, and whiteboard architectural design (not at the same time, but it could be any of these)
2) Our process is REQUIRED for a company of my size. You would not believe the amount of candidates we get. We have a high bar for hiring, but fundamentally we have to be efficient with our time. That completely removes as an option superior interviewing techniques like "spend a day with the dev team working on a small real task" or "do this take home test over the weekend and then the interviewing team will review it with you". Both of these are better at uncovering good candidates than a whiteboard coding interview, but don't scale, and are themselves controversial in the industry from candidates who are unhappy of the time they require.
3) A common criticism of "algorithmic" questions (leetcode) is that they don't reflect the reality of day to day work. If a lot of people even at FAANG are working on typical 3-tier front-end-back-end-database systems, utilizing the tools and systems other employees built before, why bother evaluating if the candidates could build the next generation of such systems if asked.
Because once these people are inside the company, we'd like to think that everyone passed a similar tech bar. That means that if a new database or distributed system does need to be purpose built for a high scale system, and someone applies internally, you can be flexible with the kind of interviewing process you put an internal transfer through - you already know they passed a high technical bar, and are likely qualified to work on the hard stuff.
4) another common misconception is people underestimate the complexity of "typical" systems inside the FAANGs of the world - compared to their prior job experiences. A payment processing integration at Amazon scale is not a payment processing integration at your company's scale. An internal CRM build at Google scale is not an internal CRM build at your company's scale.
The 10,000 foot view is similar, but the expectation would be that the system would be 1/ built out faster, 2/ built out better, and 3/ scale better than a comparable project on a less technically strong company.
5) Coming back to the interview process. What that means, is that when we ask a candidate to solve a hard algorithmic problem on a whiteboard in 20 minutes, or deep dive into the inner workings of a complex data structure, or show how they could combine multiple data structures to solve one problem, we are not validating "Can this candidate write an algorithm in 20 minutes on the job at their desk"
What we're checking is "Can this candidate do something really hard, ambiguous, and time-sensitive in the Computer Science domain, when prompted to." The real life challenges will be different, but if they CAN do the first one, they'll likely be able to do the second one.
Of course we miss out on candidates who are terrible at these skills, or can't code on a whiteboard, or are overwhemled by the interview stress and can't perform. But we are willing to do that because of Point (2) above. We'd rather say no to a "maybe" candidate than hire someone who can't perform.
Despite what people state that ability with algorithmic performance on a whiteboard does not translate to on the job success, that is not my experience. Remember - I'm not looking to see if a person can spit out A* search algo on a whiteboard from memory.
I'm looking to see how they approach a problem they haven't seen before, how they break it down, what kind of assumptions they make, what ways they simplify it, how they consider ideas and tradeoffs, and finally how their ideas translate to code (fluently or not)
6) The majority of people I see cannot execute that last paragraph well enough to join a team at my company and be productive. Sorry, hackernews readers, but that's just the case. This isn't a "can't solve fizzbuzz" level of incompetence - these are good engineers who are likely stars at their smaller companies, and who can be relied upon to get the job done. But they would not thrive at a FAANG-level company because they lack some ability - either clarity/coherence of thought, or ability to understand their techncial domain in depth enough, or because they just aren't fluent at translating ideas into code.
edit2:
7) All this to say...if you're not FAANG++(Dropbox, Salesforce, etc I don't know)....stop copying us. You don't need this process.
Your company has fewer candidates, and you both can't afford to be as picky as us, and you don't NEED to.
Each of my previous companies (in the 200-500 person range) had a process of a simple coding question onsite, followed by a take home test that a decent candidate (and I don't mean decent for FAANG, I mean decent across the SDE community) could complete a reasonable solution in ~2-3 hours. We set expectations as "you could do it in a weekend" because certainly some people took closer to 8 and got hired.
If the devs at your company need to put out fires constantly, then maybe it's a good test of whether they'll be able to do that.
1) The companies that do this interview a TON, so there are bound to be bad experiences, and those are the ones that people blog about
2) The questions are supposed to be hard, but achievable. So you get two scenarios:
A/ candidates who are not good enough and are not aware of it. "How am I supposed to know if the hashmap/dicationary data structure in the language I say I'm best at maintains insertion order? I just google that every time - everyone does!". Sorry, bucko. That's the kind of in-depth understanding of your tools that we DO expect you to give a shit about
B/ Immature interviewers with a bar that's too high that are using overly complex questions and don't get noticed right away by the more experienced interviewers.
So ultimately, YEAH, the questions I ask, for me to want to move forward with a candidate, I expect you to solve the problem. But I'm not expecting you to solve it in the most efficient way or GTFO
At a high level that reflects the work that we do. The project scale may be 20 weeks instead of 20 minutes, but the level of complexity is scaled accordingly.
Of course, we miss out on some candidates that could be great at executing on a large timeline but can't demonstrate it in a short in person interview.
But, (see my edit), this is all we can do to scale our interview process, and we don't want to take risks.
i look more after how people approach problems at a high level, where they show how they build the whole system and how they interact with peers and other departments to tell if they can gather data, organise it in specs, and then fine tune nicely build parts of a system. not code monkeys.
The vast majority at the FAANGs are full stack engineers which includes long term system design.
The comparison to a car assembly factory is so ridiculously out of alignment with reality that I can see why you chose to go with a throwaway account.
That's kind of reductionist. I upvote answers that at least attempt to propose alternatives to the current status quo. The employer side isn't the only relevant one, which is likely a stronger reason why a lot of current hiring practices aren't popular. Some only work at FAANG scale with a large enough pool of applicants, some seem optimized to hire people that waste time on code puzzles (that doesn't make them an "algorithm wiz") and others generally put applicants in a position resembling begging for a job, especially for junior roles.
Just because something works well on some quantitative metric does neither imply that it is the only, nor the optimal solution, especially in two player games like hiring.
Not sure where you read "less pay" into my comment but fine, let's go with that and just ignore that quite a few companies immitate FAANG style hiring these days.
> Instead you should try to find the good candidates they miss. However there is really no reason for top companies to change their interviews.
Yeah, I feel like we're arguing two completely different things here, there's plenty of reasons for companies to change processes even if they work on some metric. When a large number of experienced participants in the fields you are hiring for are offering critique on your process it should be at least reason enough to talk about it, which is the whole point of discussions like these.
My point was that they shouldn't unless their offer can compete with FAANG offers.
> When a large number of experienced participants in the fields you are hiring for are offering critique on your process it should be at least reason enough to talk about it
FAANG are spending billions a year to keep this process running, there are huge amounts of work put into trying to change or improve it. And some things do change, but the signals from doing algorithms is just too salient to throw away.
The longer answer is we can only study the people that joined the company and failed to perform rather than those that COULD have been outstanding but our interview process selected them out. So there are flaws, of course.
I'm not sure how you could do the latter unless the biggest companies shared their data and compared scenarios of "Candidate X failed an Company Y interview, but then later got a job at Company Z, and thrived".
Also you'd be working against the bias of "Well, that just means that the Company Z is easier to work at".
There are just too many variables, team to team.
Part of the problem is even within these massive companies there is a huge diversity of types of jobs, and an engineer who thrives on working on Google AdWords might fail on the SpannerDB team (to pick two random examples from not my company)
I don’t think the FAANG-style coding puzzles have anything new to add to the conversation here. The reason big companies do that is because the questions are prompts are simple to write explain (hence interviewer has to do very very little), the answer space is generally well-defined (so it’s easy to grade, and easy for a Hiring Manager to check if the interviewer screws up), and they almost exclusively select for completion time (which is key in a high-pressure feature house company).
Most of the discussion here seems to be interested in alternatives. Through my experience, I can’t agree enough with the article author who posts:
Not a single person out of several thousand emails and messages came out in defense of the current state of interviewing processes - I’ll let this one stand on its own.
The only people I’ve seen to defend the current process in public are ex-FAANG people who were actively monetizing their experience (e.g. Gayle Laakmann).
These are not "puzzles". That makes it seem like there is some trick or clever concept the candidate needs to grasp that show "lateral thinking".
There isn't.
These are (or supposed to be) fundamental problems of computer science that require good understanding of algorithms, data structures, code performance, and can be solved in <30 mins on a whiteboard.
Yes, these exist, and there are plenty of them. A pretty common and well known one is "HEAD to TAIL" - where you have a dictionary of 4 letter words, and given an input of 2 words find a connecting path between them changing one letter at a time. This isn't a puzzle. It's a problem to solve and approach from different angles and allowing for a large number of solutions of different performance characteristics
> Not a single person out of several thousand emails and messages came out in defense of the current state of interviewing processes - I’ll let this one stand on its own.
Means absolutely nothing. Most of us are not interested in ruining the day of someone who is unhappy with the hiring process by revealing the truth that most likely it just means the person was not good enough. That seems cruel. I'm kind of going further here though because this is the second time this OP is coming up on the front page of hackernews
A lot of people at my company do, but you never hear about this stuff because nobody blogs about a career fair that came with some friendly helpful insights from senior engineers.
Also nobody blogs about recruiters when they are super helpful and give a ton of relevant interview prep material (which ours do), partially because most candidates frankly ignore the material thinking it's not relevant.
What's more is problematic is that the construction and evaluation of the puzzles is highly subjective and nearly everywhere lacks rigor. Completion time is a key metric while quality of communication is most commonly either ignored or not assessed at all uniformly among interviews.
(Aside: what's frustrating is that companies make candidates sign NDAs for on-sites; these prevent candidates for disclosing or _selling_ the information they might glean in the process, which very rarely happens. In actuality, it's the former interviewers who violate protections for "confidential info" and copyright when they go and monetize their experience post-job, or even on-the-job e.g. Rooftop Slushie).
> Most of us are not interested in ruining the day of someone who is unhappy with the hiring process
That's only true in so far as you, as an employee, are paid to achieve positive sentiment among candidates, no matter how shallow that sentiment may be. That's indeed cruel, and it's well-established that interviewers are widely unaware of the consequences of the current hiring process. That's why we get articles like those from the OP.
A true interest in improving the hiring process includes: * Making prep materials and courses freely available (helps industry candidates) * Committing time to CSE outreach to better integrate company needs to CSE (helps new grads) * Finding questions that reflect real tasks on the job (helps the evaluation have a chance of being predictive) * Closing the information gap between hiring managers and candidates: disclosure (in aggregate) of hiring rates, salaries, etc.
Playing along and doing hundreds of coding puzzle interviews is a waste of time for all involved.
Nothing could be further from the truth at my company
> A true interest in improving the hiring process includes: * Making prep materials and courses freely available (helps industry candidates)
Recruiters share this with candidates. They ignore it.
> * Committing time to CSE outreach to better integrate company needs to CSE (helps new grads)
Go to any university career fair and you'll see companies with booths clarifying these positions working against the mis information of negative blog posts
> * Finding questions that reflect real tasks on the job (helps the evaluation have a chance of being predictive)
They do. You don't have to agree, but they do. The real task on the job is "disambiguate a complex problem autonomously, and come up with a plan to address it." The interview is a 20 minute constrained version of it.
* Closing the information gap between hiring managers and candidates: disclosure (in aggregate) of hiring rates, salaries, etc.
Semi-agreed. I do wish people and companies were more transparent about salaries. But there is already an entitlement complex from a bunch of engineers on this site and many others complaining why some dude at Netflix in California is making 500k, while he is making 80 in Oklahoma writing code for Bank of America.
Appreciate it. For some background it would be nice if you clarify the perspective of your role as one of HR/coordination or a more technical view on participants of these interviews.
Do you feel like current FAANG hiring practices for lower level roles incubate a generation of programmers that trains specifically for the interview process moreso than they do for the actual role you want to fill (and if so, do you feel like that is benefitial to your employer / simply a necessity at their scale / worth changing)?
edit: That makes sense, thanks!
No, I don't think so. We would not hire a person who clearly only memorized a bunch of algorithms and data structures to solve toy problems, but cannot describe their thought process coherently, does not show a fluent ability to translate their ideas into code, or can't refactor the code or adapt their process when introduced with new requirements, scaling bottlenecks, or system failure scenarios.
See my edit, but most interview candidates I see onsite (that's after resume/phone screens) are not good enough to join the company.
Our onsite pass rate is between 15-25%
I think the state of despair is caused by companies copying practices from FAANGs without understanding the intricacies of why those practices are in place. To the point of asking a UX/UI dev to solve a DP problem.
I think that would show the candidate’s communication style and attention to detail, in addition to their technical knowledge, and probably give a better sense of seniority too. Most code reviews are done in well under an hour anyway, vs actual dev work on a ticket that takes hours or days, so you’re doing more realistic swe work.
https://en.wikipedia.org/wiki/Bar_examination
https://en.wikipedia.org/wiki/Architect_Registration_Examina...
https://en.wikipedia.org/wiki/Principles_and_Practice_of_Eng...
We do interviews in this field like casting calls. Come on over and audition for this role. So basically we're saying 'software engineering' is a talent and not a skill.
The top comment is brilliant and sums up the brokenness in the best way I've ever seen.
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and posting more in the intended spirit of the site, we'd be grateful. Note this one: "Assume good faith." That's not because people do always act in good faith (of course that's not so), it's because the assumption of bad faith leads to crappy HN threads and the assumption of good faith leads to better ones.
We detached this comment from https://news.ycombinator.com/item?id=22393019.