I spent three months working full time to get a job
kolesky.com
kolesky.com
1. The "assignment" took me about 4 hours, which they expected and told me off the bat. 2. It involved mostly relevant code with 1-2 wrinkles that were not odd just not everyday sorts of things.
For the proceeding interview, I was able to sit with 3-4 engineers for 1-2 hours really just diving into details about my experience and their processes, etc.
So if I could offer advice it would be to find a "homework problem" your existing crew could build independently in 4 hours or so. As in you've actually had more than 1 team member attempt the request, and they did it in about 4 hours each.
This was actually an interview process I didn't mind. I have 15-ish years of experience doing development. I can spare half a day for a prospect no problem. It was actually kind of fun.
4 hours seems to be a good "sweet spot". You can ask something sufficiently technical and oriented to the daily work without being a bore. And when the interview comes around, you can make sure it's not some ego-bound contest, more of a conversation.
The crazy thing was that over half of the applicants that passed a resume screen and phone interview could not complete the homework assignment. Over half. Some couldn't get it at all (either gave up or had compile errors), some got it to run but it didn't work, a few got it to mostly work but their programs had serious bugs, and the last few actually got it right.
The assignment was to read a file containing a list of numbers (some formatted incorrectly, so there was some very simple parsing logic involved), call an API using each correctly formatted number as a parameter, and store what the API returned to a file.
I'm still shocked over half of developers couldn't accomplish that task. Makes me wonder what they were accomplishing in their existing roles.
I'm perfectly competent at writing complex code. In fact I'm usually the guy who fixes bugs or perf issues that nobody else believes exist.
A problem you came up with on your own is going to seem simpler to you than to everyone else. Why? Because you thought you picked it at random but your subconscious picked the one it thought was good. In other words, you picked the one you were already primed to answer, and it seemed simple.
So either people are going to work at it longer, the timid will give up (there's nothing wrong with timid coworkers as long as they have someone to listen to them), and the brash will get it done in the time allotment by cutting corners.
Are those the people you were trying to hire? Because that's who you're gonna get.
I absolutely would not hire someone who failed that test. Your defence of it seems bizarre to me.
I think these tests are a great tool, but like anything we all just need to be aware and careful of the biases introduced. Also, we need to be mindful that really good developers who are employed, have families etc, may not have an over abundance of time to spend on these, so if 10 companies each send a challenge that takes 4 hours, and it's unclear if the assessment will lead to further interviews, some better candidates may give up on the process or be selective about the ones they complete.
I think this creates an additional bias, that the people who complete your assessments may be the ones with the most time to invest in the process (ie the currently unemployed).
The catch? There is no internet in Cuba, so they had to do it using a book, and asking questions to a programmer who was helping them there.
It's also better to aim small (a la FizzBuzz) and walk up the complexity slowly, as opposed to starting with "Solve this ticket from our prod JIRA queue, with tests please" (which I have actually been given as a test!)
I like to aim for a very simple task, like proposed above, with maybe one non-obvious part or edge case (which the spec mentions, so it's not a trick question). Just enough to demonstrate a) clean coding for the simple parts, and b) ability to solve something without using a completely naive solution.
Uhuh.
I had a shared library running on a bunch of systems. One was VxWorks. Cross compiled. I knew nothing about VxWorks except that the code needed to run about 8x faster. I got about 2.5x out of my code by improving locality of reference problems that appeared to be intrinsic but just required experience to see. For the rest I filed a bug report to WindRiver and five to the cross compiler writer. At the end of this process I knew more about the VxWorks administration than anybody else in the group.
People give up, even when there's a clear business case for continuing.
But reading a file, doing some parsing adjustments, and calling an API sounds like a small amount of python code... so I guess not so bad :\
for num in $(egrep --only-matching '^[0-9]+' input-file); do curl -vv -X GET "http://example.com/api/v1/foo?num=${num}" -o "resp-${num}"; done
There is no incentive to cheat. Why would you want a job you are not qualified to do.
From your experience, it sounds like these are good tests, because it's screening for qualified candidates, unlike the phone interview.
money.
I am not so good with words sometimes... Haha...
Why do they do it? I have no idea.
I once asked a candidate to write a for loop in C#[0], using my laptop and Visual Studio and watched for a solid 5 minutes while he struggled -- And I don't mean jitters around syntax, I mean, failing to declare a variable properly and while he knew what a for loop was, he had no idea how to write one in C#. His resume touted a series of projects that we estimated put him a little above a Junior developer and the only language he referenced was C#. I don't normally do these kinds of "on the spot tests" because I, personally, hate them, but I was cheering when I read Coding Horror's "Fizz Buzz" article[1] on the subject.
This was the most foul example and my only experience is with developer and ops/security positions, but if you're getting half of developers who are in the same time zone as the title[2], you're doing well. Does this happen in other parts of the business world? Some of the folks we get in would be like a Walmart cashier applying to run a finance department because "they've handled money before". And what if they got the job, do they think they'd come close to being successful? I'm all for the "fake it until you make it" and I'm very willing to overlook weaknesses around experience/education if the candidate is really passionate and I can get enough of a comfort level that they'll be a quick study and love the job, but you have to be realistic about your deficiencies.
A lot of the problem, I've found (being on both sides of this in the past) is with recruiters, as well. They tend to operate on a "throw darts at the wall, blindfolded" approach, with the assumption that if they send out a million candidates for a million jobs without regard for qualifications, that they'll get lucky, one will land somewhere and they'll get a commission from sheer volume on bad odds. This serves nobody well -- it causes companies to put up multi-step gates that involve people who aren't close enough to the field to discern a good candidate from a bad one via resume performing a filter operation that disproportionately disqualifies the really passionate but "not quite a perfect fit on the qualifications side" and it causes job seekers to take a bunch of interviews for positions they won't like/aren't qualified for and won't land -- causing them to have a miserable experience interviewing (and probably resulting in some dropping out of the job market entirely).
[0] The exercise was, specifically, write a for loop with an int value that is incremented by one, adds that int value to another int variable, and multiplies the result against the counter, assigning the value to a third variable. Print the result to the console. This was before "Fizz Buzz" became all the rage, I did my best to give a blindingly simple problem to just validate that the person could do something in the language.
[1] https://blog.codinghorror.com/why-cant-programmers-program/
[2] Never mind the details "Must have 4 years of experience in Angular", nonsense -- a topic for another post, but the way we write job requirements is about stupid. I mean, I've seen experience requirements that exceed the length of time the technology was in existence, and I'd rather see a candidate demonstrate an ability to adapt and learn new things than be overly concerned with whether or not they've used a framework that is really similar to a bunch of other frameworks.
Bad employees are constantly in need of placement: people get fired very frequently.
If you're hearing about a particular job or candidate, the odds are they are bad. Most jobs are filled internally or by referral. IIRC, it's only about 20% of jobs that are filled by recruiters and/or cold job applications.
If candidates who can't code are actually getting on site, your phone screening is inadequate and vulnerable to "smooth talkers".
I'm going to have to assess come applicants soon and have been trying to come up with something appropriate, which is why I'm interested.
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400+ hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is, I should be the only one doing the assignment and there should be no wishy-washy rejections.
All of the interview processes I've had recently have checked I did the exercise by either asking me to describe what I built with no aid, asking me to walk through the code with it in front of me, or asking me to conceptually expand on what I built on a whiteboard.
Whittle down the pool even further and the ones you think look good on paper, and after an interview (or a series of them), have them come in, on a paid basis.
It'd be a sign of good faith that they're both serious about finding a good candidate, and respectful of your time?
I know some companies do paid stuff like that. If only it were more commonplace.
I did an assignment like that for a company once, I'm still waiting for the money for more than one year...
Note that 5-minute-long task is a good pre-filter on its own.
I wonder if the best thing to do is to simply offer the candidate a choice.
Why is that a bad thing? The prospective candidates aren't entitled to anything, the burden should be on them to prove they have any capabilities for the job.
> What is the employer putting up? Generally nothing.
Totally incorrect, have you ever had to hire somebody? The initial phone screens alone to weed out the 99% of complete incompetents takes many hours/days. Even getting to the list of phone screens, involves many hours of sifting through resumes, where each resume is cleverly designed to hide the fact that the prospective candidate is incompetent.
I echo the grandparent's sentiments, myself, from my own personal experience as well.
>If you want the job, you'll find four hours.
But what happens when everyone starts having these 4 hour screens? Now its a 40 hour process to apply for 10 jobs in a week (and this is before any commitment on the part of the employer). It's not sustainable or fair. And yet, there's no incentive for employers not to do it. So we as job seekers just need to suck it up and be thankful for the opportunity, right?
After interviewing 10 (say two three hours spent on each between contacting, scheduling up etc) it was clear every unemployed with 'can use word' as skill boosted their resume and applied.
I'm sorry for the real developers looking for a job but a company can't dump 20 days work on chasing and weeding liers.
I'm not a fan of homeworks myself but this is what it has come to.
I'd be payng but that puts the company and the prospect in a very difficult position them towards tax laws and towards their current employer non competes, the company toward again tax laws and then there's the absurd of paying for contract work delivered that's 99% of times not up to spec.
Beside, the unwillingness to compromise smells itself and given the choice I'd never get a potential troublemaker in that only thinks time=money and screw everything else.
That little empathy for both sided of the problems sure is a red flag.
Edit: well unless it's a contracting job then it's fine.
we used to give out this http://play.elevatorsaga.com/#challenge=3 and asking for solution to level 3 which can be done in a short time + is the first non trivial challenge.
of course solutions can be find in internet etc, but it's pretty easy to look them up if they smell fishy and it'd all come down crashing at the face-to-face anyway
it's also a very good exercise in real world problem solving and a source of endless technical discussion since there's no single good approach, especially with two elevators to schedule.
Most of the places I've been at were desperate for 'good' people and they couldn't seem to find them. It's still a seller's market out there.
If this were true people would be poached a lot more frequently than what we can observe now.
Can you, personally, pay me a salary? If you can and I want to work with you, I absolutely think it's fair to ask four hours of my own time to demonstrate I'm worth consideration for it. If not, then why does your opinion matter?
Furthermore, I already have a job, just as most reasonable candidates do - so if you want to convince me that I should spend many hours on that, you'd better be sure that your offer has a really attractive first impression and seems likely to be better than the job I currently have, otherwise you'll be self-selecting your candidates only from the (generally less skilled) sub-population of workers for whom "If you want the job, you'll find four hours" is true because they actually are desperate for jobs, unlike most skilled people in tech industry.
I have never seen a job posting I was sufficiently interested in to put in 4 hours with no investment from the company.
Regardless, if you have a good job already, why apply to any position you're not particularly interested in and that you wouldn't spend four hours trying to get?
Why would I spend any of my already very limited time if they didn't make sure the process is streamlined?
Also, I don't think it's uncommon to like what a company does and being interested without yet knowing if you're interested enough to put the time aside - most company sites and job descriptions are much too generic and vague, so I'd want to talk to some employees first (remember, interviews are two way conversations). So I might not know if I'm interested enough at the time of application, just that I might be.
[1] for my current job, the interview process was essentially them trying to persuade me to join them, and my previous job I was actively looking for a job and someone I know at the company said I should apply. My "application" consisted of telling him "ok set up an interview"
The standard process where the first two steps are some ordering of a 1-hour interview with someone technical and a chat with a recruiter is perfect for this. With minimal effort (maybe 90 minutes) you find out whether they're interested in you at all, and get to ask both a technical person and a recruiter about the job.
The big time investment typically comes later, and I can easily bail if I decide it isn't going to be a compelling offer. I've done this quite a bit.
I'm glad the homework process worked for you, but my point that it doesn't seem like a good idea for the company stands. If the first thing you do is filter based on how badly applicants need a job, you're losing a significant portion of the best people right off the bat.
So now I'm doing a day and a half of unpaid work for them, taken out of evenings and time with the family.
Not hypothetical. I just described my last job hunt, 16 months ago.
The place I really wanted to go closed the req on me, and then opened it back up the day after I started at another place. My third choice (stars didn't align on the other 2 interviews, in part because of stuff like this).
It goes both ways.
EDIT: I want to provide an anecdote around this. If your salary is $150k, it may cost you roughly $400 in gross pay to interview somewhere. Sure, you could take paid vacation, but vacation time is paid out in cash when you leave so it still costs you. Companies with unlimited vacation are a different story.
I had a similar experience a month ago. It was a similar "4 hour project" but it was open ended with "make it as complicated or as simple as you want". Being a single table backed CRUD app, you could only put in so many capabilities but it got me excited thinking about ways I could surprise the staff. In the end, I made it self-hosted (in a language that self-hosted is less common), included an installer, and made it support multiple connections from different devices with updates that pushed to the connected clients. The project took me about a day and a half (with two of those hours spent yelling at WIX), but the core parts took about two hours. I was genuinely surprised at how much fun I had putting it together. They also asked me to estimate my time spent -- since I had contract work I was doing at the time and couldn't dedicate a long stretch all at once to it -- I broke it down into its individual tasks and documented the whole thing. They had no problem with the fact that I delivered it a week after the interview.
I also second your "I can spare a half a day for a prospect" attitude. There's a flip-side benefit to that, as well. A candidate who feels over burdened with a simple assignment (provided it's tested as you mentioned and truly is simple) is probably not one I'd be interested in hiring, anyway. I'm not saying it's "one size fits all" -- I've had candidates walk in the door that I knew in advance were really good fits who were hired after a 30-minute screen. That, though, is a unicorn sighting.
If 10 good assignments meant at least 3 offers, it'd be a different story, right?
In particular, it seems to me that take-home assignments are strictly better for candidate scheduling etc than in-person interviews, if time spent can be held equal. Maybe time spent is just not equal very often?
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400 hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is to say, I should be the only one doing the assignment and there should be no wishy-washy rejections.
Unless you've been in the shoes of a hiring manager, I'm unsure as to where this idea is coming from?
Having been a hiring manager, it is a horrible experience. Unless it's extremely naive, homework sets generally take time to come up with and formalize, and certainly take time to review the results for.
I suppose I can understand conflating the motivations of an interviewer with their employer, but it's not generally true to say that interviewers are just trying to exploit those they interview.
It's a one-off effort to come up with an assignment. So the cost per application can be extremely low. As far as evaluating them, the bad ones can usually be tossed out in minutes or less. Comparatively, the cost per applicant is basically nothing.
There's nothing wrong with that, it's a competitive market, those who get hired are those willing to compete, not those who complain about unfairness about being asked to demonstrate their skills. Employers spend tons of time filtering candidates, even reviewing the submitted work takes time and effort, saying it's no cost to the employer whatsoever is not only untrue, it's naive.
What?! Up until that point, typically, the employer had crafted the job posting, posted in on various channels, reviewed the responses, talked to promising candidates, and then give you the assignment. And a good assignment needs to be carefully crafted, a process that usually takes an order of magnitude time more than what it takes to prepare an answer.
I think your last paragraph is very close, but I think the thing that matters is "no wishy-washy rejections". Not being the only person taking the test or how much time the company spends seem like ways to make that happen to me.
Maybe I'm just completely wrong here, but I think most of the objections to homework type tests are being caused by processes where the homework test was in addition to the in-person whiteboard interview, not instead of it.
How would you feel about a process that was: some preliminary screening -> nominal 4 hour homework thing -> in person meeting to confirm that you're not a complete asshole and negotiate salary?
It is a sellers market. I get recruited every single day. Any company that disrespects me and wastes my time isn't going to be very successful at convincing me to work there.
Developers are the ones with the power in this market, and I have no problem throwing this power around.
Maybe just open separate channels for people who have an open source portfolio. I have about 30 repos in my OSS, but still have to take these assignments. Its such an utter waste of my time.
That is a huge red flag, and means that they will disrespect me at work as well.
So that's an absolute floor of an engineer-hour per candidate: Anything less than 30 minutes to read through it and take notes is useless, and figure another 30 minutes minimum for a call.
If they're not going over the responses in detail and then talking through them with the candidates, then they're just wasting their own time because they're not actually judging anything.
And yet companies have an incredibly hard time hiring talent, even with so many other fish out there. Odd, that.
People need to start understanding their value as an employee. You create more value than you're worth; that's why companies need workers. You don't need to bend over for a potential employer. Plenty of other fish in the sea of employers, too.
And I am honestly just an average SF engineer.
The current engineering market let's me choose whatever company I want.
So, I am not going to waste my valuable time with companies that disrespect me, when there are a hundred more that I could be talking to instead.
If the "homework assignments" are just a scheme to get work done for free, or a competent engineer's time is not set aside to review them, then that's a different story. But as a fair test of programming ability, administered in good faith, I don't see the problem, and such a test is important.
Ask to see the company's employment agreements or contracts in writing first.
I was once given a 10 hour assignment, which indeed took that long to complete. Afterwards, the company expressed interest in bringing me on, and sent me a work agreement to sign, which included many unsavory terms such as being paid for completed work 90 days out.
Since their terms were shit, I didn't take the job, and lost 10+ hours of my life.
Overall level was pretty low that after reviewing about 20 homework assignments, and I had wasted a lot of time on post-mortem interviews where I just had to politely decline with a short code review (after all, they spent all this time, so they deserve something in exchange). After all, I started asking FizzBuzz (yes, not changing the name or anything) on the first phone interview to save time. And yes, turns out, it's not a task that any candidate can do.
So, yes - homework assignments are great, but make sure that a candidate at least knows how to write a simple loop (and yes, I overlook off-by-one errors).
The kicker though, like someone else below mentioned, is that we would receive submissions that didn't even compile.
What I liked about the HW problem is that even something simple gives you and the candidate something to talk about. Why did you do this or that. Tabs or spaces ;) etc...
Remember, the interview goes both ways. You can look dumb for your choice of questions just as much as the interviewee can look dumb for their choice of answers.
- The homework assignment wasn't open and public: it was an option after doing a brief (30 minute) phone interview.
- The other option was basically coming in for onsite technical interviews.
As imperfect as ALL technical recruiting styles may be, at least a short coding assignment (i.e. < 4 hrs) is halfway-related to the real world. I have written code against a set of requirements nearly every day of my professional career. Outside of the interview context, I haven't balanced a red-black tree since I was in undergrad over 15 years ago. Frankly, whenever I hear people favoring the latter over the former, I always assume that it's just because they're fresh out of school themselves.
Yeah, doing dozens of homework assignments for multiple companies could be a massive time sink. However, I've almost universally felt like I wasted my time when going through the interview process with companies who don't use them.
Perhaps my perspective is a bit colored, as I'm far enough along in my career that I can be choosy about which companies I invest time in. I only talk with a dozen or fewer companies in each job search, and so I won't do a 4-hour homework assignment unless I'm already interested in working for you. If I were closer to entry level, and therefore had to go through the process with a hundred companies to land an offer, maybe I'd have a different opinion.
Regardless, the original post here is some entrepreneur, writing a small novel about his lengthy process of figuring out which job title to seek at a real company ("CTO"? "VP"? "Architect"? "Principal"?) Of course it took him 200+ hours to get a clue.
My time is valuable and if an employer isn't willing to give equal effort in the interview process, I'm not interested. Also I'm not particularly interested in a workplace – especially a startup – that insists on this in a recruitment process. It leads to only hiring employees who are willing to play the game which is much more likely to exclude people who want to live lives outside of their job and fosters a culture of obsessive overwork as well as not having the diversity of opinion/demeanor/background/etc. which is valuable in a healthy workplace.
If you want to bring me in and spend a day with me talking and working in your office, fine. I won't, however, spend 4-10 hours working on a problem for you while working at the job I already have and doing the rest of life.
I would much rather do a homework assignment and then come in and have a code review or design discussion about my solution than do the standard process. And, I would argue that it gives the company a much better look at my skills and aptitudes than the whiteboard interview. I have never once written a dynamic programming algorithm from scratch, in 11 years of programming. I have implemented graph theory algorithms only a handful of times. But create a relatively basic piece of functionality, and then justify my solution in code review? I do that literally every day, and sometimes more than once per day.
I understand that writing code for a homework assignment can be seen as giving the company I'm interviewing for "free labor". But, as others have pointed out, from the interviewee's perspective, there's not a huge distinction between spending 8 hours cramming Introduction to Algorithms and Cracking the Coding Interview versus spending 8 hours making a website using Django or Ruby on Rails (to name two interview tasks that I've actually been presented with). Frankly, I was much happier writing the websites, as it seemed much less pointless than memorizing how to write Floyd-Warshall all-pairs shortest path in Java on a whiteboard.
1) I've been coding forever, and have faced code reviews in multiple professional environments. So your "6-8 hours" will probably only take me less than two, and my code will probably look better than yours to a reviewer.
2) I get tired of having to re-read "Cracking the Code Interview", because almost none of that is frequently used outside of academia. And dealing with smug 5-year journeymen interviewers, who are accustomed to asking 1-year juniors to traverse binary search trees and name all of the design patterns that they've memorized from books.
No matter what process your company uses, there will always be people trash-talking it on the Internet. "I refuse to interview at those places", "I insist on being paid an hourly rate to deal with them", etc. I'm sure that 99% of this is chest-thumping bullshit. But regardless, if you're going to filter out part of the candidate pool no matter which path you take... I'd much rather filter those 25-year olds who feel too entitled to write some code, and legitimately believe that companies should just glance at their GitHub page and then hurl them a giant sack of money.
Wanting the employer to spend "equal effort" is silly. Their loss is not necessarily your gain.
Ironically, one of the things outside interests can teach you about is something called overtraining, where you can actually get worse at doing something from doing it more or harder.
If you've heard cross-training in a fitness context, it's named that because it's intended to be an antidote for overtraining. Branching out makes you more resilient. Your ability to solve problems isn't helped by hyperfocussing on an extremely narrow context.
Any inventor can tell you that the really big stuff comes from fusing ideas from seemingly unrelated disciplines. And yet we bury our heads because we don't know any better, and we don't know better because we've buried our heads.
A willingness to give away your time for free is an indication you don't value yourself that highly.
Nobody is asking for free work. And I definitely would never hire someone that calls a four hour exercise that is part of an application process "an indication you don't value yourself highly." In fact, it would be a huge red flag and I'd throw that applicant'a resume into the trash can.
If you can't see the symmetry in the situation it says something about your values, but it is not a compelling argument.
Interesting. I've found the opposite. The farther I get in my career (and the older I get--not going to say the "A" word), the more difficult it is to get through the interview gauntlet. Companies looking to fill Junior "5th engineer from the left" roles tend to be fine with a warm body and a brain. Companies looking for super-senior-staff engineers, engineering managers or even more senior folks are extremely picky and probably reject almost everyone they meet. Even when I'm not actively looking for work, I try to answer every recruiter E-mail try to reach out to 1-2 companies a week just to keep irons hot and be able to act on the off chance that something spectacular pops up.
The older you get the MORE shoe rubber you need to go through pounding pavement.
Yes, but what it is not is economical. There's a huge dropoff in the interview funnel at this step because it's largely not worth the effort for the interviewee. The choice is between spending 5-10 hours per take home test per company or spending 5-10 hours a week studying silly whiteboard exercises that apply to dozens of companies.
It's also just as uneconomical for the interviewer. I've completed about a dozen of these tests over the last year and only one company bothered to look at the test after I sent it in. Which makes sense; it takes time to look at a lot of code, figure out which of it is boilerplate and which of it was written fresh for the test. And even then, there's no guarantee somebody else didn't write it. And while you may not be interviewing with hundreds of companies, most companies easily get hundreds of applicants per position these days.
For whatever reason, it doesn't seem to be worth anybody's time to do these tests. This bums me out because I'm terrible at live coding tests. But, this is the reality of the situation, so I have to adapt.
FWIW, I've done a total of 1 for each of the last 2 job searches I've been on, and ended up getting an offer out of each one, so it seems perfectly worthwhile to me to do them.
All the take home tests have been given before I had any phone screen with any engineer. The point is to save both your time.
It's worthless if you have the test after already wasting an afternoon on the phone.
Which company asked you to balance a red-black tree in an interview context? Name-and-shame them. Seriously. I've heard many whiteboard interview polemicists use this example, but have never seen any evidence for it actually happening.
Hey, I googled for it, here it is: http://programmingzen.com/things-ive-learned-from-hiring-int...
> the question is to explain how the red-black tree algorithm works.
From your link:
> Sample questions included: What’s a Red-Black Tree and what would you use it for?
I don't see how those are equivalent.
I also don't see how the latter two questions are equivalent to asking an interviewee to "balance a red-black tree".
while homework problems can be better than in-person interviews, they can also be a lot worse. for one, because they are NOT time-bound. secondly, they are much more difficult to setup and evaluate: do you provide a spec? code to modify? third, asking people to do work for free in order to explore the potential relationship is a big ask. lastly, they are very time-consuming for the company.
in grad school i learned the hard way to always ask for in-class exams instead of take-homes :-). time and scope constraints can be favorable both for the company and the candidate.
> Enough with the graph traversal problems.
the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill. it is that they are a way to gain information about the candidate's problem-solving ability in a limited amount of time. unavoidably, these questions test for speed and familiarity along with problem-solving. that makes them a lot less than perfect, but i've generally found them valuable.
After the interview he said I was clearly weak on math because I didn't know the definition of a square root. Something I did know (10 or so college math courses in linear algebra, calc etc) and was describing, but was at a loss for words.
Asking to work somewhere without demonstrating that you can do the work is asking a lot. Unfortunately, job history alone is not enough to weed out people that cannot program (let alone well.)
> they are very time-consuming for the company.
Not if you implement a simple test spec or scoring system. We're engineers!
> the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill.
So don't ask the generic questions. Ask someone to implement something that they would need to do on the job. Obviously don't ask them to implement a feature that takes 12 hours, but a basic screen that is a snapshot of 45 minutes of on-the-job experience isn't crazy.
We also do a take-home exercise first to see how you think without us staring at you. Day-to-day at Seneca Systems we aren't staring at you, so it feels more relevant.
Which other professions have this attitude? I've either worked or supported a pretty wide variety of other jobs, and I can't off the top of my head think of a single one that does.
Even if some examples are brought up (and I'm genuinely interested in what they'd be), it's far from the usual approach in most hiring situations.
I generally prefer an existing portfolio of code (e.g. github repos) to see the kind of code someone produces, but if you aren't able to show some fairly extensive examples of your work, homework is usually a bunch better assessment of your skills than what can be accomplished in a ~45 min interview.
When I wrote my comment, I was thinking of the medical profession, dentistry, auto & other mechanics, appliance and other device repair, construction ("there's a pile of lumber in the back, you have an hour to build a shed and then we'll consider hiring you"), plumbing, civil engineering ...
There are a lot of clowns out there, on both sides of the table. At this point I've had my quota of companies that sounded good in the interview process but ended up being like a bad second date. Full of chaos and not anything I needed in my life.
One place drilled me on testing, and I mistook that for caring about tests. Nope. It turned out that not only did they not have any test automation, but their code base was such a ball of mud that you couldn't have many of them if you wanted to. I had absolutely no interest in pushing for a renewal on that contract.
Um, this is a good thing, not a bad thing. Research shows that stress significantly reduces cognitive ability, and time pressure definitely adds to stress - sometimes substantially. What this means is that when you ask whiteboard coding questions during the interview, you're not measuring the candidate's on-the-job ability. You're simply measuring how well they perform under severe pressure. If that's fine with you, as in it's representative of your work environment where people have to churn out code around the clock, fine. Otherwise, homework questions are much better.
Also the stress of someone standing over your shoulder watching you work. At work, I enjoy hashing out problems with a coworkers at a whiteboard, but in an interview it's paralyzing to have someone watching me. As a student, I'd totally freeze up in the middle of a test whenever a teacher started walking down my aisle.
but also, keep in mind that the candidate has a limited amount of time, and often a full-time job on top of the job hunt. the homework questions are only good if the candidate has time and desire to them.
I find this point interesting, as to me, this would be an argument for, not against. Time constraints on homework assignments are a lot closer to the constraints you'd have programming for a job than "whiteboard coding" would be. I'd hazard a guess that doing a good job on an in-depth, open-ended problem given a few days to a week is a better indicator of ability to be a successful software engineer than being able to solve a logic puzzle on a whiteboard in fifteen minutes.
i agree that it seems easier to simulate working conditions with a homework assignment. (it does add a lot of work for the company to do homework problems well, but let's leave that aside)
the downside is that the candidate has a limited amount of time to talk to you. they often have a full-time job going on top of the job hunt. secondly, are you really game to spend 0.5-2 days of unpaid work just for the chance of being invited to interview further?
An additional perspective I didn't think of until now is whether this type of recruitment process would be more beneficial to the industry as a whole. An argument could be made that we'd all be better off as a whole if interviews tended to more accurately assess candidates and candidates were able to get a more realistic sense of the type of work they'd be doing for a company.
Actually, my codebase has an honest to god graph traversal in it, on the front-end no less. Basic manipulation of graph relationships without much friction should be standard.
I've only experienced this once and it was stressful as could be, but was from a person I absolutely respected and still do. No-bs, either right or wrong and plenty to discuss in the followup conversation.
Did it from home, pulled the file at around 11pm right when I knew my best time to work was. Would happily do as part of an interview any time.
For example you can document one module and say "the rest of the docs are TODO due to time" - it shows you know how to document and you can move to another bit. My favourite submissions did 5% of many things to show they are aware of those areas, rather than super-polished trivial solutions of just the basic question.
This is the truth and it hurts. Take a company with 20 job openings, each of which you are probably semi-qualified for, and could see yourself doing. As a candidate, you're limited to applying for one of them, and if you strike out, you're back to square one--often below square one because companies make you wait N months before applying again.
I'd love it if a company were to say, "Hey we're going to pass on you for this job in Division X but you know you're smart and Division Y's role is a MUCH better fit for you--why don't we hire you for that?" I've never seen this happen in my life.
Try talking to any company's recruiter and say, "You know, I specifically want to work for your company but am not sure what the right role is. Why don't we do an interview and then determine where the best fit is" and watch their brains explode. Nobody does it that way, but wouldn't it be so much better?
EDIT: Another note, at least three of the roles I interviewed for had homework assignments. None took longer than 2 hours and I actually enjoyed them and the subsequent interviews much better.
I've had it happen twice (almost 3).
I just find a company, look through the listings, find a job I like then send an email explaining my qualifications and my interest.
I think it works well if the company is small and roles are fairly undefined ("You aren't an exact fit for this, but we want your skills for something else")
For companies where the number of spots is significantly dwarfed by the number of applicants, probably not - you either increase the levels of indirection involved between "initial contact" and "job offer/not" in order to try to add bin-fitting tiers at the beginning, or you significantly increase the burden of interviews by requiring them to be able to judge for multiple different types of role.
For smaller places, it's the "nobody ever got fired for buying IBM" syndrome - rather than trying to fit based on what the prospective managers/departments want, the initial pass is for ass-covering so that HR can't be accused of negligence if a candidate doesn't work out because they looked perfect on paper. Of course, then these places are left wondering why they have open positions for years...
Small enough companies (e.g. where they don't necessarily have full-time recruiters), I've found that your example at the end actually can fly.
Edit: Why are people scared of whiteboards interviews so much? Don't you use them every day to map thoughts out, working with your colleagues etc etc? I'm not saying some questions are (used to be) absurd - the Google/MSFT manhole thing. But why is it so scary to talk about a linked list, a graph traversal algorithm etc? Most of this is stuff that's taught in basic CS classes.
Sorry to sound condescending but it almost feels like that as technology spreads and more and more people come into the field who haven't actually studied proper CS, they're blaming the system instead of trying to learn.
That's just unfair, inconsiderate and unsustainable, and is also a red flag (ie: I would not go to that company).
Giving up a week of your life for an interview ?
and for free ?
And most importantly - what if ALL companies did this ? you would have to spend months working from home, for free, just to get to the next level of the hiring process .
I don't know what the solution IS, but I know for sure that it IS NOT a 1-week work-from-home-for-free task .
(edit:formatting and some grammar)
Edit2: people will actually pay good money for a yelp (or craigslist/etc/etc) clone on places like upwork. And they asked you to do it for free .
it is probably something like : 1. go to upwork
2. find a 2-3 hours task (eg: scrape entire site [sitename].com, design a logo etc. etc.
3. give the same task to 5 different candidates
4. choose the best one
5. make profit and gain upwork cred
6. send a "thanks but we chose a different candidate" email to all the candidates
7. repeat
(edit:formatting)
edit2: you can even let candidate X make a code review on candidate Y.
step 8: automate the process and go public
Edit: It's ironic how the so called cutting-edge tech companies can't afford a keyboard, mouse, laptop and a projector which would make things way better. I even had a case where the interviewer changed the question with 5 min. to spare , I explained the solution he said it would work and then told me to write it. I was just erasing and pointing arrows because I did not have space etc. So easy to forget that while typing you just hit enter and get a new line. Draw on white board, write on computer; that's how it should be.
I won't discount your theory of a bias, but I think it is a small influencer if it exists.
Absolutely agree and I'll add that you're shopping in a market for lemons. There are multiple adverse selection criteria against the pool of job seekers. (Excellent engineers are more likely to be retained/less likely to be fired. Excellent engineers are more likely to receive an offer when they do interview. Poor engineers are less likely to be retained, more likely to be fired, less likely to be hired.)
In steady state, the applicant pool is of lower overall average quality than the pool of employed engineers.
Our place has a 12-month contract before being made perm. Once you are perm you are not getting let go unless it's part of larger cutbacks (government). He was made perm before 12 months and I was astounded. They didn't even ask me what I thought before they did it.
Well above avoiding mishires, loyalty concerns are paramount in every organization I've worked for. You'd think that they'd treat hiring as the quintessential bet-the-company decision, and operate accordingly, but this simply doesn't happen. Leadership creates policies and procedures designed to introduce opacity in the process with the ostensible goal of avoiding mishires, but the real goal of preserving power.
Can you quantify? I read this all the time but nobody explains it.
Meanwhile, this talking point is used to justify an atrocious waste of interviewee time and ridiculous interview requirements.
This only costs actual cash outlay, not opportunity cost of other team members (could be coding but are instead spinning up the new engineer), overhead from management salaries, cost to redo work of an other-than-productive employee, etc.
At Valley salaries hiring someone for a day costs you $100k++. In more moderately compensated locales, lower bound it at $40k or so.
Sorry, WHAT?
>>pportunity cost of other team members (could be coding but are instead spinning up the new engineer)
Hey I've got news for you. Developers don't just waltz in and start coding from day 1, they need to be brought up to speed on the systems. Your business-nosed attitude needs a reality check. Developers are human capital, not robots.
You're being unnecessarily defensive. Just because people are people does not change the fact that it costs companies money to onboard them. This whole subthread is "why is it expensive to hire people?" and one part of the answer is "because you have to pay them money while you train them, and pay their colleagues while they do it". It's not a reflection of any unrealistic expectations--it's the opposite.
> Sorry, WHAT?
Computer, desk, office space: $5k
Job ads: $500-3k per hire is not unrealistic
Time spent sifting through applications: Easily 5 hours per week. If the posting is up for a month, that's 20 hours. That translates to about $1.5k in fully loaded costs for an employee with a $100k salary.
Time spent conducting phone screens: Assume 50 phones screens for one position. 30 min per person (20 minute conversation, 10 minute notes/break). 25 hours. Approximately $1.8k for the same $100k employee.
Further interview rounds: Assume about 10 candidates per position. Assume at least 4 person-hours per candidate, plus at least 8 hours devising evaluations (programming tests or what-not). 48 hours = ~$3.6k. Add in $1k of travel costs per candidate = $10k.
Draw up an employment offer. Assume you've got a stock one, but the new hire would like to make an amendment. Run it by a lawyer. $500.
We're at about $25k. It's not $40k, but it's not free, either. It would be easy to spend more time on any of those steps. If you use a recruiter, it will be higher. If you calculate based on SV salaries and costs, it will be higher.
If that's the figure, then maybe bad hires really aren't that bad. Presumably the company won't have gotten zero or negative value from a bad hire, and even if it was $0, $25k isn't going to make or break any but the scrappiest bootstrapper.
2. opportunity cost of actually hiring someone good - by the time you've realized you've made a mistake, a good person is off the market.
3. you actually have to pay a bad hire for their time. you can't just not pay them. that's cold hard cash out the door.
4. everyone on the team starts wondering, how the hell did this guy make it through this process? are we being run by idiots?
5. if the hire is remote in another state, you have to register your business with their tax authorities, and deal with that whole payroll rigamarole.
6. also health insurance bullshit.
7. setting up accounts, changing passwords/keys when they get canned.
8. actually firing them is not fun unless you are a total sociopath, or they are actively causing damage by destroying value (which is extremely rare -- most people will just silently not do a fucking thing for weeks on end).
9. worrying about a lawsuit because ironically, people who cause damage like to threaten these things.
this is on top of your already full workload of doing actual, productive things for actual paying customers and good employees.
I don't know how else to do it, especially when it is really rare for someone to have the skills we need out of the gate. That means testing (we have a testing process) and time talking with candidates. On the other side, streamlining things, we have made hiring missteps in the past, which had measurable and high cost to the business. For us at least, it isn't hot air. Hiring is a hard problem. We spent a lot of time trying to optimize it to find an ideal way that costs less time.
Consider this: every hour a consultant is interviewing they arent doing research or billable work. So we believe strongly enough in how we do things to put our money where our practices are. Maybe the economics for the Googles of the world are different, but for us it is a real decision to be made.
I don't think I've ever heard a manager even say the phrase 'turnover' until they have a few ex-employees that they really couldn't afford to have lost, and a bunch of disgruntled people still there. Once the horse has left it's too late to close the barn.
If it gets me good people to defect (make my hiring easier), I win. The world won't copy me so once I hire people, I can still keep them.
I don't do this because I don't think I'll get good people.
Individual Germans may not have thought of themselves as particularly bigoted against Jews, but that didn't stop them from participating in regimes that marginalized them.
Most of the people fighting against slavery owned slaves. Jim Crow persisted well into the sixties and it's probable that many of the people against slavery would have been for Jim Crow.
It's very possible for the hiring market to be biased against liquidity even when everyone wishes it were easier to hire.
Note that Jim Crow/Davis Bacon/etc was a legal regime, a "must discriminate" set of laws to prevent people from acting in their own self interest. What such mechanism exists today preventing me from acting in my own interest?
I think SWE as a profession is in a little bit of denial, because we honestly (want to) believe we are being efficient. But the reality probably is that most SW companies don't do very useful work, but rather, struggling to make ends meet, are fighting tooth and nail to prove (often through deception and introducing complexity) that somebody needs their software. And this of course in turn affects the workers in these companies.
So SWEs will have to admit (mostly like everybody else) that it is actually a game of musical chairs, and if they don't want this game to happen in this fashion, they will have to institute some other way to play this game (of proving that you are useful), for example, by having a profession association and a profession exam.
Think about it. If there would really be that much work to be done in SWE, the companies would just take everybody. But they don't, they are more and more picky. There have been other articles linked from HN (such as http://spectrum.ieee.org/at-work/education/the-stem-crisis-i...) that say the same thing.
Here in Prague, situation is a little different to US (from my reading). SWEs are locally in quite high demand, as we are on the other side of the globalization. So companies often take almost anybody (for a reasonably paid job).
On the other hand, if the demand is stagnant, you will try to get the best people by being very picky in attempt to lower the costs.
I have seen E1, E2, SE and PE used as abbreviations for level of experience (Graduate Level 1, Graduate Level 2, Senior, Principal). These levels might have been applied to a RE (Reliability Engineer), SE (Safety Engineer), EE (Electrical Engineer) etc
So there is already potential confusion in the abbreviations used for various engineering roles. SWE provides some disambiguation. Interestingly HWE seems to be used for Hardware Engineer - when they are generally Electrical Engineers.
I like Triplebytes [1] focused approach to hiring engineers. I think they used to interview candidates to assess their skills rather than depend on whiteboards or resumes. If the candidate passes the interview, they would be connected to relevant companies, take up interviews with the companies and negotiate the offer(s). Now it looks like they changed the process slightly to introduce online coding tests to make it easier to screen more candidates. Edit : As pointed out below, the online coding test is the first step followed by one on one interviews.
http://blog.triplebyte.com/12-000-engineers-evaluated
If you don't make it to the interview phase with Triplebyte, then the process might as well be as opaque to you as any company. You aren't given much information about how well you tested and the reason you weren't selected to interview.
It would be great if there were a Triplebyte-like service for those of us in the lower 10,000, but until then it's back to applying to dozens of companies.
Here are a few things that kept personally me from completing the screening process (which I was checking out because I was a recruiter, and now I'm a developer):
- You don't (or didn't) support Elixir, which is what I've primarily been using for the last 12 months, so I spent a few minutes dusting off the "what's the syntax in Ruby" knowlegde
- I'm wasn't emotionally invested into applying, as I don't really know who you'll be connecting me with, and if I want to be challenged, I'll continue working on some personal code (which is what I did)
- Not having my glasses with me, and a bit of the ol' ADHD, made reading just a bit harder, so the countdown timer was basically reminding me the entire time that I couldn't read fast enough
> Still getting programming homework interviews
Kill me.
I was happy with my solution, and disappointed when I was rejected.
Rejection is fine, but what upset me more was radio silence when asking for any feedback on my solution. As a young engineer it would have been amazing to get feedback from professionals.
The reason I ask is that that kind of behavior, where you click on the node and it highlights the paths to that node but grays out everything else, is super useful for complicated architecture diagrams where you want to put a ton of stuff in one diagram, but still be able to see where edges go when you have a big spider web of connections. I've been looking for diagramming software that would create these clickable kinds of diagrams for my company, but haven't found anything that exactly fits the bill (Most tools I've found generate static diagrams, and some mind mapping software I've seen doesn't totally fit the bill, either).
I did some work on something a little bit similar for an EU research project, which you are welcome to take a look at. It's probably not directly applicable, but it may give you some thoughts about what does and doesn't work as a UI: https://github.com/cse-bristol/process-model/ https://tools.smartsteep.eu/process-model/?name=actionplanni...
I don't get this. I'm in my early 20s and have professionally implemented graph traversal algorithms at least five times. Graphs are extremely powerful data structures, and if you have them, then you can expect to traverse/search them.
Funny story(now, wasn't at the time but been a few years now). Made it to a final interview at one of the "elite" tech companies.
Was interviewing for a web development position, and the second whiteboard question was about designing a hash table. I of course was confused as I had no backend knowledge. Next one was about a graph traversal, and again I was confused.
So, because of that I was scared shitless that I needed to know that stuff and went out/crammed it.
Ended up interviewing other places, got an offer and didn't have to answer it.
However, because of that "elite" company's questions I went and learned more about graphs, queues, stacks etc.
So while I was pissed off at them for awhile for wasting my time, in the end without them I might not have spent the time learning and progressing through Comp Sci topics.
As a front end developer, if you're consuming APIs and building out interfaces, and passing data back to the APIs you don't have to know much about graphs or hash tables.
For most web developer/front-end positions, you can be perfectly fine without knowing much of the "advanced" topics of computer science. You're going to be designing to the product manager's specs, paired with a back-end programmer who is going to tell you the structure of the data that will be passed back down and then you build it from there.
As far as understanding hash tables or graphs, you can really get away with knowing them.
However, I agree with you after learning these topics. There's incredibly more efficient ways to build front end pages with the concepts of a hash table/lookup. Most recently I designed a page that basically uses the URL slugs as keys in a complex dictionary to grab content from a SQL database to render the pages.
In addition to that, it's also just wonderful to be part of the conversation when you go to design APIs or architecture so you can make sure to relay/point out concerns that may cause a lot of unneeded pressure on the front end.
Tl;dr You can be a web dev or front end without much knowledge of hash tables, graphs but I can't imagine you advancing your career very far without that knowledge.
You have to deal with these kind of structures anyway. You have to be able to render such data as hierarchy of components, you have to be able to get some information from it, to transform it, to update it.
Tree traversal is almost as common in front-end world as using .map, .filter or .reduce, I think (and using `reduce` also can seem hard on beginning, but it's quite useful).
This very statement reminds me of famous "why can't programmers program" article: https://blog.codinghorror.com/why-cant-programmers-program/
Programming needs some technical skill anyway. And learning "how to use library X" often takes more time than learning "how to implement algorithm X". Libraries are often bloated, bugged, hard to debug or have terrible API. In these moments basic programming skill set is very useful.
But I think in the long run everything depends on exact project rather on our wishful thinking.
I had experiences when use of library was much better solution than own baked one, but I also had experiences where none of the libraries I had found were quite fitted for the project and many times I just had to throw library and write something from scratch in simple way.
> reusable libs are tested by the whole community.
reusable libs are often FOR the whole community and not for YOUR project. Ofc sometimes you need exactly what rest of community needs, but sometimes not.
It's clear to me that they had never intended to offer the senior position and just used it as bait.
Other times, they said I would be interviewing with the CEO but when I got there, they put me through some standard recruitment process and I didn't even see the CEO.
This destroys the startup's credibility in my eyes. I cannot work for liars. Making me do all these tests before telling me what I was really applying for shows that they completely disregarded the value of my time.
I prefer this kind of tests, as they only require an hour of effort, and they are objective. All the time a technical test was involved I received a quick and good offer. When I had to convince them about my skills just by talking I either got no offer or I got an offer that was way to low for my experience (like half or worse).
As an example, the hardest problem in the test was a C function that receives two parameters, one by value, another by reference, and it did some modifications to those parameters. The problem was further complicated by having two global variables with the exact same names as the parameters that were used to call the function but with the variables switched (global a went to parameter b, global b went in parameter a). At the end they asked what's the value of global a, b?
*Disclaimer: All numbers are estimates. Product can be something you work on internally for a big company to battle office politics or externally when doing your own thing.
Convincing people that what you do is righteous takes time.
We are unpredictable, some are great at deception, we have massive mood swings and phases where we are not at our best, we can deteriorate rapidly and get distracted. What is supposed to be a stable reflection is our education degrees and experience and ideally coupled with a few conversations this should be enough.
The problem here is companies have convinced themselves they can find the perfect candidate for every hire and pursue this unicorn relentlessly. Even with all the ado they will frequently get it wrong. It would be more mature and productive to loosen up and deal with the risks than subject everyone to the kind of process.
I'm not bitter about it, just crossed it off as a learning experience and moved on, but every time I see one of these articles about interviewing for tech roles, I'm constantly wondering what the best process is.
The big 4 tend to hire the best they can get, and have enough applications to pick from a wide talent pool so can afford to spend time doing whiteboard interviews etc, but for other organisations? I'm not so sure.
I didn't collect metrics like this, but it took about a year and something north of 170 applications/reachouts. I received zero direct rejections. Probably actually talked to about 15 different companies. Phone interviewed with 3 or 4 and interviewed directly with 2.
I ended up not getting any of those positions.
But I don't feel bad about it, the positions where I made it to the interview rounds weren't even the ones I applied to within those companies. Both of the interviews were friendly, but almost entirely pointless and were non-representative of what I would have likely been doing there anyway.
Finally, after really wanting to leave, I just did the time honored thing of asking a few friends who worked at places I was interested in if they'd refer me. Two places said they'd like to talk, and I work at one of them now. This process took about 3 weeks and I ended up in as near a perfect fit as you can get.
Side-effect, after working at this new place for a couple years, those friends I asked, keep popping up every once in a while to see if I'd be interested in something where they currently work...so they've kept in mind that I both:
- switch jobs
- thought of them when I was looking to do it
I'm not a developer, I'm much more into Technical Project/Program Management, so YMMV, but I also noticed that the hiring processes for people like me have almost no steam behind them. Most companies tried real hard to fit me into developer or engineer pipelines, they had almost no ability to entertain anybody outside of that kind of hiring experience. Of the few that I phone interviewed with, at some point the subject of doing a coding assignment came up and I simply told them that we'd all be sad at events around that idea:
I'm a passable coder, but years out of date. I haven't coded professionally for probably getting onto 15 years now. The job I wanted didn't involve any software development, so the exercise wasn't very useful. If I passed it, it both said something about their engineering maturity (and I wouldn't want to work there) and again wouldn't be representative of my work.
The actual hiring machine at most companies seems impossibly broken. I ran into a few good recruiters, but most were just trying to make quota or show effort.
This mirrors the experience I've had at several jobs, so I don't think it's terribly rare.
Where it's been different than this has been when getting referred to your everyday startup -- and usually those processes are so braindead I end up in a queue for a programming exchange that won't matter in the end. And I've interviewed at some of the biggest "startups" that regularly make front-page here -- so I'm not talking about weird edge cases.
Yeah, that's the kind of company I was referred to. I assume you don't work in startups? :)
> Hours doing homeworks: 33
I would prefer this to be the opposite: more homework, less talking . That's what the job is going to be like, anyway ..
(edit:Formatting)
I have this near conspiracy theory about why the interviewing process includes such ridiculous things as balancing red black trees. Like every other regulation/tax, the propagation of crazy ideas about interviewing practices is really self serving for the tech giants. In a large orgy of stupidity, companies which can neither afford the impedance of nor are particularly good at these CS type questions mimic the practices because they are probably trying to show that they have a high bar.
Unfortunately, the multi-billion dollar companies can churn through the candidate selection process and waste more money in the process than many other companies' annual revenue. But I am sure they realize the halo effect of making a great deal of noise about this practice, because this is one of those strategies with very little downside and all kinds of unintended upside.
If I were a recruiter at one of the top 10 companies, I would spend half my time creating fictitious accounts on various social platforms to ask about the CS question du jour at these interviews. It would be the equivalent of click fraud in tech job search.
On a more serious note, I wonder if companies trying to hire these employees ever really do a cost analysis of the interview process itself? At what point does it stop making sense to "just wait till we get that truly exceptional candidate"?
I really worry that if someone tries to establish a software engineering system, then that someone will be unreasonably confident, and probably wedded to a particular method or process. Then we'll all be stuck having to prove we know UML (or the equivalent 2017 fad) inside-out in order to get a job.
Anyway, isn't this what university degrees are supposed to be doing? Maybe we just need a more effective way to shame universities that graduate computer scientists who can't program?
Further, that standard commonly used algos and data structures that make a good design are established and basically a standard part of every modern language. What does seem to be rapidly changing is the framework du jour where for some reason we keep reinventing the wheel thinking that we are doing things better when in actuality we just keep creating competing frameworks that don't necessarily benefit anyone. I'm all for competition but eventually a clear winner is defined in a space and it might be better to contribute to improving that winner than not.
Finally I'd say software is in a better place now than medicine was when it was about the same age. We are much more evidence driven (but still lots of opinions persist). And as with any other scientific discipline (because let's face it that's what software is) it will change with new literature so of course we should have a continuing education component.
The reality is, the only party that doesn't benefit from having a credentialed software practitioner is companies because now they have their pick of the litter and can control prices. But if all of a sudden the barrier to entry to becoming a software engineer is high and only licensed practitioners can allow code to be released into the world, we become an expensive resource.
I think what would really help transform the hiring process is to assess skills that would be required on the job. I think that was the spirit of the original submission here, but I just wanted to explicitly state that I don't think homework is the solution to remedying the current state of hiring in all cases insofar as actually assessing the skills that would be required on the job.
These unaligned 3 screens-long pictures with sections all over the place is a mess.
I can't tell if the layout is just weird on purpose? the site is buggy? or adblock/noscript are screwing something?
That could be abused. If the assignment takes longer than, say, an hour - it's unfair to have someone waste their time, especially if every prospective job has one too.
With a homework assignment that is not timed, you let the candidate do her best possible work in her ideal environment, and yes, while a work environment is sometimes chaotic, when she needs to get her work done, she is going to make her environment as perfect as possible.
At first, I was a little less enthusiastic about the approach -- I have GitHub repos and other examples of competency out there -- but as I thought about it, this "test app" was in line with what they were looking for and when they described the approach, I liked it. The first thing they did correctly was provide a basic spec sheet. It wasn't overly specific and I think this was by design. It was basically a CRUD web app, with a statement of "feel free to get creative; do as little or as much as you feel it needs". I provided a self-hosted CRUD app that worked on multiple devices, syncing state between them in real-time, documentation, an installer and everything else that was needed to make it a complete product. Most of that wasn't on the spec sheet, but in line with thinking about the project as "something I'd deliver", those were things I'd consider to make it complete and it also gave me an opportunity to demonstrate competency in different frameworks, provide examples of functional/OOP design and other areas that gave them an adequate look at what I could do. The app, itself, was a really minimal web app that took me a couple of hours to write, but after all of the additions, it landed in at a day and a half or so -- not a terrible amount of work for someone out of work and worth it since I landed the job.
Frankly, given the choice between shoulder-surf coding, whiteboard coding[2] and test projects, I'll take the test project. I hate whiteboard interviews and thankfully there were none of those this time around. I've had two whiteboard interviews in the last five years. In one case, I got lucky in that the first algorithm was one I had written into an app I had on a private (personal) BitBucket repo, so I pivoted with "I can marker up your board here or I can show you a good implementation that I did a few weeks ago". We spent the rest of the interview with them asking questions about my choices and that was one of the most enjoyable interviews I've had. I didn't take that job. The other, I didn't get as lucky and actually ended up turning down the job on my way out of the interview room[2].
I agree with the author about having empathy for the job seeker. This empathy, though, works both ways. In my case, I had filled in for a dev team in a pinch -- at the last minute -- to run an interview. They hired the guy on my recommendation and he ended up being a great employee so I was pulled in to most of the interviews for several of the dev teams from that point forward. Never mind that I did development on an ops/security team and didn't work with any of these teams, making whether or not they hired a particular person -- or anyone at all -- of now benefit to anything I was responsible for. As a guy who had an already overbooked week, I didn't enjoy being pulled into these and I really didn't appreciate the massive number of woefully under-qualified candidates or the fact that all of that time was often spent on a great candidate who turned down the job in the end. Because of this, I express thanks at various times during the interview process and specifically mention how grateful I am for the time taken out of their already busy work-life. I was glad for the opportunity to interview at all of the places that chose to interview me so I simply made that known. The cool thing about that is later interviews felt more at-ease with the team as a result[3].
[0] I had quite better luck than the author, but did fewer interviews, and was seeking a position that was right in line with my experience.
[1] I like this idea, too, provided compensation is provided -- after all, if you intend to use the product of the work a candidate performs, they're technically a contractor and I wouldn't do work like this for free.
[2] I actually do pretty well on my toes -- I'm good at rationalizing out loud, and breaking up the tension with a little humor (to the point that I've been given hints at answers when I was stuck). Despite turning down this job, they made a really good offer to me two weeks later that I declined -- the job itself was in a domain I wasn't excited about, but the whiteboarding didn't help.
[3] Felt being the operative word -- these were later interviews which means the team was already more interested in me as a candidate than in the first interview. I also remember the first time a candidate specifically thanked me for the time I spent interviewing him and very likely projected that onto my interviewer. But at the same time, it increased my confidence so if it was all in my head, I'm OK with that. I also feel that it was "the right thing to do".
It is as from Perlin's foreword to SICP:
"The computers are never large enough or fast enough. Each breakthrough in hardware technology leads to more massive programming enterprises, new organizational principles, and an enrichment of abstract models. Every reader should ask himself periodically ``Toward what end, toward what end?'' -- but do not ask it too often lest you pass up the fun of programming for the constipation of bittersweet philosophy."
I am stuck in this constipation, because I feel lonely, and I feel the computer is driving us to lonelier and lonelier ends, and, as much as I feel terrible hatred at my lack of employment, I would rather play with you than work with you if our work really is making the world lonelier and more and more dependent on anti-depressants to make up for how we have somehow missed something precious, perhaps glimpsed by Harry Harlow, in our mad dash to try to replace terrycloth and humble fare with wire mesh and our cornucopia of junk.
The thing to do is to remove proxies for competence and intermediary parties. Talk directly to the candidate, about the actual work (don't try to "see how the handle" odd questions).
However, this requires decentralization of decision making which reduces control.
I tried zooming in, but even just 110% zoom breaks the text.
"I found the process to be dehumanizing, stressful, chaotic, inaccurate, opaque..."
Then goes on to talk about pigeon holing, other issues, and some suggestions.
You could criticize his conclusions or suggestions, but I don't see how you would summarize it as "not good for some reason which I won't go into here".