The latest trend for tech interviews: Days of unpaid homework
work.qz.com
work.qz.com
Yet, we do. Ok, then what about open source contributions (if they have them)? Well, sure, but we don't really know, for sure, if that code is their own or if they indeed truly wrote it, can we? So we can't count on it. It helps but it can't be an automatic "this person passes the technical" signaler.
So maybe the person shares actual code and walks us through it that they wrote, even if not open source? Again, we can't know for certain it is their own code and not some friend who wrote it for them 2 years ago that they claim is their own. Or was a teammate's, or what-have-you. So we can't count on that either.
Apart from getting a candidate to actually pump out some code, even trivially, we don't really have a way to de-facto verify that the person says they can do what they say they can do, short of a personal reference from someone, say, in the company already that knows and has worked with the person in the past.
I absolutely hate this practice of having to constantly "re-prove" to the next employer that someone knows how to write software. It's extremely redundant and yet we keep on doing it, with no real hope of actually making it better.
Don't be silly. GPDR is going to save the world, solve all of the internet's problems and bake you nice warm croissants in the morning.
Many participants in the game unfortunately insist on hiring and marketing by three-letter acronyms.
- People starring your GitHub repository.
- Someone following you on Twitter.
- Someone linking to a blog post you wrote.
- Someone recommending you to someone else because you did a great job on a project.
These are all examples of informal networks of trust.
I fail to see how that’s a bad thing.
I have 0 stars on Github. Don't use twitter. Don't have a blog. Only a few coworkers I even talk to.
I spend my time writing code and interacting with my family, not trying to win popularity contests. Now if you're looking to hire celebrities instead of programmers, you're maybe onto something. I'm not a self-promotion artist. If I were, I would work in advertising.
That kind of aversion towards self-promotion might be a reason why we arrived at absurd phenomena like homework assignments for interviewing processes in the first place.
With no way to draw upon previous work results or third party trust how else is someone supposed to assess a candidate’s skills?
If I come across or meet a new person from this industry and that person has a Twitter account I routinely check the follower count.
A somewhat large number says to me: “Many people seem to think that person has something relevant to say.”
Is there racial or gender-based inequality? Yes, unfortunately there is.
Should we therefore resign and stop trusting each other altogether? No, we shouldn’t.
We should rather try and extend our trust to others.
Inequality isn’t caused by trust but by a lack of it.
>Should we therefore resign and stop trusting each other altogether?
This is a strawman, that was not suggested. What was suggested was to have an interview process which doesn't involve informal/subjective trust. The best programmers don't have huge twitter networks or github stars - some might, but not all. That's no criteria for filtering.
I don’t think we can magically solve inequality by trying to fix the interviewing process or by denying there are social networks.
http://www.abc.net.au/news/2017-06-30/bilnd-recruitment-tria...
”Professor Michael Hiscox, a Harvard academic who oversaw the trial, said he was shocked by the results and has urged caution.
"We anticipated this would have a positive impact on diversity — making it more likely that female candidates and those from ethnic minorities are selected for the shortlist," he said.
"We found the opposite, that de-identifying candidates reduced the likelihood of women being selected for the shortlist." The trial found assigning a male name to a candidate made them 3.2 per cent less likely to get a job interview.
Adding a woman's name to a CV made the candidate 2.9 per cent more likely to get a foot in the door.
"We should hit pause and be very cautious about introducing this as a way of improving diversity, as it can have the opposite effect," Professor Hiscox said.”
Less automated / efficient hiring practices could also contribute to higher developer salaries, which would in a roundabout way, provide some compensation for what seems like "uncompensated" time spent intensively interviewing.
Does this fully replace the onsite? If I'm fully employed, interviewing with three or four places, and expected to take a day off and visit each of them, adding another full day of work on top of each is a big hurdle.
To keep going with the devil's advocate: If one of your competitors says "ok based on your initial chat with the director, we're gonna bring you straight to onsite" and you say "do this ten hour problem" guess which I'm going to favor? A hiring manager confident in his ability (probably learned through trial and error) to sniff out bullshit is a positive signal to me, the candidate. Intense homework tells me one of two things: they have way too many candidates for the role and there isn't enough data to screen by hand (this makes sense for seasonal stuff like entry-level, not so much elsewhere IMO), or they don't really have a solid picture of how to interview. And in the latter case, that probably means if someone else on the team or in the company suggests a bad but speciously-attractive technical design, they're also going to be unable to call that out.
One of the most mind-boggling interviewing processes I've been through went like this:
1. Write a program that satisfies the given requirements. Be sure to use the specified design pattern they explicitly requested. Include a suite of unit tests and instructions for building and running the program. Do this within one week.
2. Phone call with the team to discuss your program. Explain why you did certain things in a certain way. Discuss the design patterns you used, especially the one they requested.
3. A "coffee meeting" with some of the team members to discuss the cultural fit. They described their culture and the environment, and also talked about what I looked for in a job, my motivations and what I would do in certain hypothetical situations.
4. Full day of on-site interviews.
The reason I call it mind-boggling is because this process wasn't outlined beforehand, so that last step came as a total surprise; at that point, I had assumed they had enough information to decide. When I explained that I couldn't accommodate them within a week at least due to my current job, their proposed solution was to instead do two half-day interview rounds after normal working hours. This is where I decided to politely drop them, not so much because they wanted to make me go through an interview loop after a full day's work, but because they were going to make their employees do that, too. I was already concerned about the adversarial nature of their work culture -- anyone can nominate anyone else to be fired because "they don't belong there" -- and this just convinced me that I wouldn't fit in there.
The moral of the story is: your interview practices send a very strong message about your company. Be careful what that message is.
Wow. Reality show style office management, anyone?
If you keep hiring duds, you shouldn't be allowed to hire people. In spite of what the headhunter firms and cookie-cutter websites would have you believe, interviewing and hiring is a skill, not a punchlist.
>If a 10 hour project-specific coding task reduces mis-hires by 50%+, isn't it better for both parties?
Only if you pay me for my ten hours. Otherwise, you've already indicated that you and your company don't value me or my time.
> Only if you pay me for my ten hours. Otherwise, you've already indicated that you and your company don't value me or my time.
My favorite way to hire is short term consulting project into full time position. Not every candidate is willing or able to do this, but it has produced the highest success rate in terms of hiring signals. By the way, this underscores that it's not a money problem - hiring the wrong person is a lot more expensive than paying five candidates to produce something meaningful (and find the best one of the five).
Some of them were former coworkers who leaned very heavily on algorithm problems and whiteboarding because they were terrified of people who couldn't code at all (though wouldn't that be the easiest kind to let go?). But they still missed sometimes. Both on people who'd studied purely for algorithm questions and got through, despite having little practical ability; and on people who could knock out a task but had no ability to reason about its place in a larger system, so created more work for others than they contributed themselves.
In theory, yes, and that's why they're popular. But you're missing that:
1) The candidate has to do this for each job they apply for.
2) There's no investment on your end, and I have no idea whether you're just going to ignore it after I'm done.
For candidates with many years of experience, obviously they can and have been writing software effectively, so for me it mostly comes down to whether they could work well on a team, interact with clients successfully, and be capable of mentoring juniors.
I wish that were true, it'd make hiring so much easier.
This despite that my best known essay is titled "Object Oriented Programming Is An Expensive Disaster Which Must End" a fact the interviewer should know if they'd looked at my resume or spent 10 seconds looking up my name on Google. Wikipedia now lists me as one of the critics of OOP -- https://en.wikipedia.org/wiki/Object-oriented_programming
I'm lucky in that my position is comfortable, so I can laugh off stuff like that, and either educate the person I'm talking to, or end things quickly by explaining that there is probably some kind of cultural mismatch.
But it shows a remarkable laziness on the part of the people who, in theory, want to learn more about me.
I think you're overthinking the feedback you got.
If it was at one of the big names, they have the luxury of being picky. They can't hire every qualified candidate, so they have to reject most of them. As the interviewer, the company policy likely dictates the interviewer give a reason. Which puts the interviewer in a bind because it implies that there was a good reason for rejecting you. So they'll put canned statements like what you got.
I recently got rejected. And one of the reasons given was something that 3 of the 5 interviewers had explicitly told me they don't care about. I recognized it for what it was - they feel obligated to give some reason, so they nitpick after the fact even though they told you they were not going to nitpick.
The question is, can you whiteboard? They will be testing that. If you can, then all is golden and you probably didn't need to waste those 4-6 years getting that PhD because what you will be working on will have nothing to do with your topic of research (unless it was AI/ML).
If you can demonstrate that you understand programming concepts and can learn, then it's not much more to assume you can write code. If you need someone who can hit the ground running with some specific technology, then you might need to go deep into it, but not to find out if someone can program.
When I was younger, I often wondered how it was that New Senior Guy was totally incapable of actually programming, but very good at handwaving and bullshitting, and being patronizing. Now I know how those people got hired. Thanks.
You might as well have said, "Well, you know, when I was a young girl I once had a crush on an older man, and when he rebuffed me I badmouthed him on Insta and all this talk of older men coercing young women in employment is probably just schoolgirl crushes, broken hearts and vindictiveness."
It's true that young people can be cocky and arrogant. But there are unscrupulous people who use this meme to maintain their position, usually to the detriment of the young people. If your interview system is unable to identify such people then you will create a hostile environment. Hopefully, your good people will leave. Unfortunately, many will stay and blame or doubt themselves. I was very lucky in two cases to be at a start-up with a strong CEO who wanted to see results (which I could produce, but which the bullshitter could not).
I was also lucky that in those two cases, the CEO correctly interpreted my emotional expression, which ran the gamut of anger, isolation, helplessness, victimhood, righteousness, denial, negotiation, not as a lunatic but as someone in a fundamentally awful position, and why.
So in one specific case I'm thinking of, the senior guy in question tanked two consecutive projects and lost a major client, while in another, the CEO eventually called him out by asking to demonstrate doing himself what he claimed he was mentoring his team on, and firing him when he failed (the guy handwaved, boasted, and prevaricated but couldn't actually do what was asked). That CEO was fucking great.
If the industry wasn't so full of lying incompetents, it wouldn't be an issue.
I've been rejected enough to eventually learn all these. Then finally when I got hired, questions were the very basic define static, inheritance etc.
I knew it was a matter of numbers and should not mind the devastating feeling of rejection. But damn, you can never know how it feels experiencing tons of rejection until you do.
Here's the thing tho': my CV is honest and my track record is pretty good: some small companies and some name-brand, well respected in their industries, known for probing interviews, over about 20 years. There may be some bad actors in the global candidate pool. But seriously, asking me these questions is like tearing my CV up in front of me and calling me a liar to my face.
I am not in the market right now and hope not to be for a good while, but I dream of being asked a stupid whiteboard question, solving it, then throwing the board eraser at the interviewer's head and walking out...
Just skip them and ask interesting and difficult ones.
If a company cannot be flexible and reasonable enough to skip basic questions it's a bit of a red flag.
I've been having to give some interviews lately (though unfortunately they have to be within 'process') and my basic question is to write some code (with a computer) that outputs the endpoints of a line segment which bisects a plane between two input control points. (I give all the equations needed and the code is done within an application already set up.) It's still managed to trip up some people with experience who seemingly don't have the concept of representing a line mathematically (BS or MS in Physics, how?). We also tested it on intern candidates under shorter time constraints and a couple internal people, they did much better, I think all but one actually got the basic line done within half an hour even if edge cases remained. If we're given longer time constraints the problem and application framework scales to more interesting subjects like generating n-point voronoi diagrams and their applications, but we haven't gotten there with anyone yet...
I've interviewed many candidates. It happens to find people that cannot code at all.
Yet, one does not propose trivial coding challenges like FizzBuzz. Very little is gained from finding out that a senior developer can do FizzBuzz and then fails at any more complex coding. You simply go for more complex questions straight away.
Given how Da Vinchi had a resume its amazing we still use them. (His was better thoigh, it had his name and address, dates he worked for people and their names and address)
I wouldn't go that far. It still weeds out the honest-but-unqualified with minimal effort - it's basically a good early filter. This actually works for the candidate too - how much would it suck to 6 hours of interviews and then get told you don't have the right degree or some such?
Hire using NN days contract-to-perm. Make the permanent offers after NN days.
I mean if you were at a party and some guy came up to you chatting, claiming to be a senior whatever at BigTechCo, you'd start chatting him up and you could tell if he was full of BS pretty quickly.
I've literally seen someone who did firmware for the space shuttle grind for 45 minutes on fizz buzz without making progress. Like, I'd they struggle for ten minutes or so, I take a step back and say "Ok, screw syntax, let's just vaguely talk about what needs to happen and mock up some pseudo code.". Even then, grinding for another half hour with no progress.
And yet, you have to really wonder ... what did that person do writing firmware for the space shuttle? Did they really do that? Or did they somehow lose their mind after they left and now couldn't do a simple set of modulus operations on a FizzBuzz test? How can that happen? Even reading this kind of thing makes me totally scratch my head.
Plus the pressure of someone staring at you.
Then you feel stupid for not answering straight away.
The you get self conscious and start doubting yourself.
What was the question again?
I don't think there is anything especially unusual about it at all.
I worked in a different contractor in NASA/Clear Lake at the time but even before I read those papers the organization responsible for the shuttle firmware was held in the highest regard.
But people are already crying over take-home tests being like working for free for the company, and then there's no way to prove they aren't cheating, which is a showstopper. That by itself is definitely not a "better suggestion for filtering out people who can't code".
> all of these options will make the interview process more appealing for everyone.
No for people who can competently handle an onsite interview and don't want hours of extra work at home.
> And if the justification for angry/combative/aggressive interviewing is "well, that's how our culture is", I would say the company has a bigger problem to fix.
Maybe you being so disingenuous makes you feel better, but interviews aren't "angry/combative/aggressive" just because someone asks to see some code on a whiteboard.
In my experience, the whole process goes a lot smoother when there's a take-home exercise. It turns the in-person interview portion into a technical discussion about a recent mini-project unencumbered by NDAs and trade secrets.
If you can't quickly determine if an applicant "cheated" on the take-home exercise through a simple discussion, then you may want to recuse yourself from performing interviews.
If an applicant can't make enough time for an appropriately scoped exercise, I think it's likely they're unqualified or they don't value the opportunity enough to make time. Those are undesirable qualities the recruiting/interview process is intended to filter out.
I hope to never work with you.
He wasn't going to come to any works nights out or even lunches and like you said certain accomodations have to be made, e.g. access to a private quiet room for him at certain times... Able to work from home when he had to etc.
His father had to go to his interview with him, luckily the company allowed this. We have since gone our seperate ways but wherever he is working now got a great developer. I'd happily work with him again.
This is one of the biggest things I try to guard against. Some people just interview better than a lot of other people, but that doesn't mean they do anything other than interview well.
I'd prefer to do a lot of that at home personally.
What happens instead is that you just don't ever hear back from candidate.
It also shows the company is semi interested and not just meeting the consideration requirements before selecting an already chosen candidate.
And even if it's paid, it's still a not-insignificant time investment for the applicant. They're probably applying for several gigs and take home assignments would quickly turn into a full time job, and filing the tax paperwork could take more time than the coding.
And finally, assigning a job that would actually get used at a company pushes up the complexity level, the need to understand the context. This would also make me very suspicious about the motives of the company.
In my company, we give a "fizzbuzz" level assignment on the whiteboard (or their own laptop if they prefer). The purpose is to weed out the applicants who can't code at all (makes a surprisingly large portion of applicants). Additionally, we've noticed that the ones who are good programmers will ace these tests.
I don't think that whiteboard assignments or takehome work is a good way to assess how good a programmer is. A 15-30 minute smoke test with a binary result (the applicant can or can not code at all) is good to filter out bad applicants but not to distinguish good from excellent.
There are plenty of people telling you the same thing, you just don't want to hear it. Here's a data point. I would never do it unless I was absolutely desperate. I mean about to lose the house desperate. There are just too many other companies around.
I mean if you are trying to pay junior rates and maybe pull a decent mid-level guy, it may be a good tactic. If you are at all interested in top talent, you'll turn many if not most of them away. Unless your company offers something no other company in the area offers, tops generally have no time or patience for that, especially if they just handed you a loaded resume with tons of references.
If you're just a boring ole company like all the other companies, believe me, you're turning top people away.
I've never applied to Google and I never will, because it would be nearly impossible for me to perform in one of their interviews.
But I did get the job.
No expectations? No stress!
It sounds to me that an assumption is being made because they didnt meet the expectations within a fixed time period. The danger with this is that this is not a real world scenerio of how two people would approach a task or problem. It’s also important to recognize that everyone is different with regards to how they process information. Stress has a real biological effect on how the brain processes information, and who doesnt feel stressed during an interview.
I’m somewhat autistic and struggle to process verbal information. But I try not to share that with people because I choose not to be labeled by it. It’s unfortunate that in our society that we have to conceal things like that.
All of us could be better at trying to understand people around us and accepting them for who they are instead of scrutinizing them for what they are not.
Of course, you have a limited amount of time to interview people. Please let me know when you figure out how to give someone an infinite amount of time to answer questions during a 1 hour interview.
> The danger with this is that this is not a real world scenerio of how two people would approach a task or problem.
That doesn't mean it's not a reasonable way to test basic skills. It's even too easy to test that by itself. What good alternative do you have?
> All of us could be better at trying to understand people around us and accepting them for who they are instead of scrutinizing them for what they are not.
How exactly would you structure a 1 hour interview so that it's not, "scrutinizing them for what they are not"? Or are you just having fun being preachy here?
Here’s the thing though, you are “leaving money on the table” when your interview process deselects great programmers because of a mistaken belief that Test A is a good proxy for developer talent. When HN developers are telling that Test A is a bad proxy for developer competence, perhaps, just perhaps, you might consider trying to improve it with the advice given.
The upside of being able to accomodate developers who “lock up” in stressful interviews, is that you now have access to great developer that are (according to your competitors) “impossible to find”.
> you are “leaving money on the table” when your interview process deselects great programmers because of a mistaken belief that Test A is a good proxy for developer talent.
That's true for any non-perfect interview process at any job. The problem isn't that people don't know this, it's that it's hard to accurately measure who's good.
> you might consider trying to improve it with the advice given
Hah, as if the advice is something amazing! Which of these pieces of advice could we use that wouldn't make things worse in some way and also have a chance of being accepted by existing coworkers and managers? There's some great pie in the sky ideas, but nothing without serious issues.
> Pete Holiday, an Engineering Manager at CallRail in Atlanta, used to use homework as part of job interviews before realizing that he was ruling out good candidates. Some told him they didn’t have time for homework. Others may have never gotten to that point. “It’s way more inclusive to just have someone come to the office and talk to them,” Holiday said. “You’re not counting on them having time, or a computer at home. We have candidates with sick family member, single parents. Without the homework we can cast a wider net.”
So is homework worse than whiteboarding onsite? Who do we believe, some article or the crackpots here?
Think of ways you can de-stress the candidate while extracting useful information. Ideas:
1. Have them bring something they wrote with them and describe it in the interview. A lot of developers will “come out of their shell” when they start explaining something that they worked on.
2. Show them some code and have them explain it to you. I had a manager open a C book once and had me explain some code.
3. Pair program on a problem that the interviewer doesn’t know the answer to. This situation is a lot more like a true work situation.
That's what people do, or should be doing, along with the whiteboarding. I don't think anyone just has them do fizzbuzz and that's it, like you seem to be implying. That won't even fill an entire interview slot.
> it seems like it is much more enlightening to hear about how they organize a schedule...
Sure, and that's also something you ask about, but that doesn't weed out the people who can talk the talk but can't really program, who do exist.
> But even as I type this, I'm wondering if the ability to read people is why this so hard for software interviewers...or maybe it's for lack of that simple skill that our industry has resorted to silly proxies like fizz buzz, etc.
I think it's because people want real evidence of skill, which is pretty easy to detect via whiteboard in most cases, not just the interviewer's gut feeling.
Not if the fixed time period is 45 minutes with an interviewer looking over your shoulder.
Yes, and they're troubleshooted by people who are already intimately familiar with the systems they're troubleshooting. Nobody in the real world writes an app for a requirement they've never seen before from scratch in 45 minutes.
So you're asserting interviewees shouldn't already be familiar with simple problems like reversing a list? Come on, give me a break.
> Nobody in the real world writes an app for a requirement they've never seen before from scratch in 45 minutes.
Being asked to reverse a list in 15 minutes isn't even close to "writing an app from scratch". It's not even theoretical - just look at all the people who make it past these interviews. I doubt they would call it "writing an app from scratch". Could you try any harder to misrepresent things here?
Though it is clearly a simple problem, I've not had to reverse a list in any of the code I've written in the past 20 years. So I am not 'familiar' with the problem. I think I'd be able to solve it pretty quickly under normal work circumstances (even crises), but as many have said here already, working on something you've never done before with three strangers looking over your shoulder is a whole other dimension.
Many here and elsewhere have related the 'brain lockup' effect. Take me as another data point. It's real and it totally wrecks the interview for the interviewee and the interviewers, who lose out on a potentially good employee.
What's your point? You just want to repeat what everyone else has already said? And how low do you think the bar should be? What would be reasonable questions that actually still separate people who can code from people who can't, since reversing a list is too hard according to you?
> It's real and it totally wrecks the interview for the interviewee and the interviewers, who lose out on a potentially good employee.
Or they hire someone else who might actually be just as good or better. It's not like companies have an unlimited number of open positions.
I'm not saying anybody should lower any bars. Just don't immediately fail somebody who struggles with a simple problem. Don't just automatically declare them incompetent. Help the candidate be the best version of themselves. It's in your best interest as an employer. Try to figure out ways to make your interview atmosphere as realistic as possible. Some of these people who choke on trivial problems may have aced the same test on a better day.
It also isn't even close to "solving a real-world problem". Wow, look, I reversed this list and the conversion rate on our sign-up page went up 200%!
> Could you try any harder to misrepresent things here?
It looks to me like you are the one misrepresenting things.
It's not supposed to be the same thing. It's supposed to test your coding and problem solving abilities, in the limited time you have. What better alternative can you come up with? It's easy to just complain...
> It looks to me like you are the one misrepresenting things.
Hah, yeah right. If it makes you feel better, sure.
The second group universally cannot perform under production pressure.
The funny thing is an artificial examination style setup totally freezes me up. I'm a valued contributor and write original technically non-trivial code day in and day out.
The last time I did an interview a few years ago my brain completely locked up. The interviewer was very polite, and it could have not have been any less stressful situation. Yet, somehow my brain knew I was in an interview, and it was time to lock up all that valuable technical information ... somewhere where my conscious mind could not access it.
Which is really weird. Usually at work the occasional unexpected stress put's my brain on turbo-charge, and ... it's so beautiful to experience it, everything has perfect clarity, I know what needs to be done and I just do it so much faster than regularly.
My "on-interview" brain is complete opposite of my "million-euros-are-at-a-risk" brain at work. I have no idea why.
I have 20 years of industry experience. Everything on my resume is the truth. I've worked on serious code, for serious projects, at serious companies. At work, my performance reviews have been stellar.
When I do live coding interviews, I'm being tested with tasks that are utterly remedial relative to what I've done professionally over long periods of time. Yet I flub these remedial tasks. It could be because I'm nervous. Could be because the interviewer keeps interrupting me mid-line, so I can't think. Could be they are too terse, and I can't drag enough information out of them. Could be the problem is slightly more complex than I can accomplish under any circumstances in a live performance, but that I could complete correctly in a closed room.
These interviews are degrading, disrespectful, and they tell lies to the misguided interviewer. What do they think they're testing the candidate for? Why does it matter whether you can code something remedial, live? Is live performance part of the job function? Do other professions do this? We should end this practice.
Getting the right team is one of, if not THE key to startup success.
Yet you consider it not worth a senior engineer's time to evaluate the candidates for the next hire, and the presumably with whom they will need to work - productively?
Check your org's runway and be sure to have your resume ready when it gets close to the end. I wouldn't expect take-off.
I wish you luck
Literally every other profession has this challenge! Programmers are unique only in their seeming inability to realize that they are not unique.
Accountants are not asked to solve double-entry accounting whiteboard problems. Finance professionals are not given take-home banking projects. Only programmers seem to be unable to be reasonable interviewers.
No, this is a made-up difficulty, caused by the fact that programmers think there’s a silver bullet solution to evaluating competency in an “objective” manner.
One major proctored exam scales a lot better than every company for themselves reinventing an almost proctored exam.
https://engineers.texas.gov/downloads/ncees_PESoftware_2013....
Certainly what I see in the PE today seems like a good starting base. Having some idea of Big O complexity can be a great place to start when working with algorithms, and certainly from my perspective I'd rather have an engineer with a solid idea of Big O complexity trade-off issues than a bunch of rote memorized implementations of classic algorithms.
Also, don't forget that a lot of discrete math and algorithms bubbled up into the Fundamentals of Engineering test that is the prerequisite of the PE test:
https://ncees.org/wp-content/uploads/FE-Ele-CBT-specs.pdf
Which isn't to say that the PE can't be improved for Software Engineering to better meet industry needs, but certainly the current PE still stands as a good starting point that the industry could try to hone to a better tool if they wanted.
My girlfriend has just graduated college with an Accounting degree.
> There's nothing like it for programmers.
What exactly is the difference between a BA (Accounting), and a BSc (Computing) in terms of validation?
Now if you want to talk CPA? Sure, but the equivalent of that would be a Software Engineering (similar to PE/CE/ME).
Apples and oranges.
Or... large companies, particularly defense, excel at hiding mediocrity in their ranks, and it's easy for someone who produces no or negative work to be handed around rather than fired.
What's the success rate for startups?
The reality here is that everyone is supposing their particular bias explains what happened.
No one seems to have asked what actually went wrong in the interview. Was the interviewee truly incompetent? Were they nervous? Were they fine at code but terrible at interviews? Was the language or environment unfamiliar? Had they just got off a plane after zero hours of sleep?
Without data, opinions are worthless. And generally, there's too little data about the relationship between interview performance and job performance, and too much unthinking mimicry: "We do this because everyone else does."
> What's the success rate for startups?
While there are certainly mediocre coders in many many startups, let's put the blame for most startup failures firmly where it lies:
"It was a turd of an idea and no-one told the founder/the founder didn't believe it."
not
"It was an amazing revolutionary concept that was only sabotaged by dimwits who couldn't fizzbuzz."
Going through a series of technical filters is not a problem in and of itself. The problem is that each company has its own set of filters, many times with skills that don't necessarily transfer to other interviews.
With the "bar exam" approach, you only need to pass 1 exam for N number of companies, leverage that to fast-track through a company interview. With the current approach, you need to take N exams for N companies, and return to square one if you fail. There is no extra layer of validation that lets you keep a "ahead" placement in these interviews. Company specific interviews should focus on basic soft skills, and if they mesh well with the team.
I'm an experienced programmer- not yet considered senior level, but have worked at several companies -and having a hell of a time going through my latest job hunt.
I have been applying for jobs for 3 years straight now, and it's telling that the only offers I've gotten were three short term contract gigs- one from a past employer who's already familiar with my work, and two from small studios with a very informal process where they just looked at my resume and Github projects and with that, they were very convinced that I'm the man for their job. I took these jobs mainly to add more skills and to fill up the gaping time hole in my resume.
But for the "regular" full-time jobs at businesses, big or small, local or on the west coast (I'm a midwesterner), I have not gotten an offer. I've been through many tests and live coding going through a lot of things. Triplebyte expects you to be a speed demon, but at least they give detailed feedback on what you did wrong.
So what is with the saturation of applicants and tendency to make the hiring process slower and slower? That's what these coding tests and interviews do, they make the process slower.
It's not 2009 anymore- unemployment is reasonably low, so this current phenomena of rigorous testing shouldn't happen. More programmers need to discover and embrace the power of collective bargaining.
Either there are a lot of companies which employ unskilled developers who cannot even code fizzbuzz, or interviewing is not a great way to understand specific types of people.
But I totally failed a technical screen.
It’s been a week of reflection as a result. I nearly canceled the interview to begin with because the entire process felt like a cattle call - totally cold, the company had shared no information about themselves or the role. There were ten minutes of rapid fire (“you have one minute to answer this question.”) followed by the proclamation that the next 45 minutes would be spent on “as many coding problems as we can get through.”
Within the first few minutes I was able to describe a general solution to the problem, but over the next 40 I totally struggled to produce a working solution. Now, sitting here I could likely write it in 5 minutes. The problem was:
--I was completely frustrated by the experience from the beginning
--I was in a code editor that I wasn’t familiar with
--I had limited access to documentation
--An interviewer looking over my shoulder harassing me
--I realized that while the problem was simple, there were things I’d just never needed in that particular language before, so didn’t have a complete understanding of what was available to me.
--I focus on design before implementation, where the interviewer just wanted me to jump in and bang out code.
--I realized that the last time I wrote this particular bit of code was 20+ years ago, as a college freshman, and certainly not in the language I was using today.
So I agree with the other commenters: we should all check our premises before we disregard another experienced programmer as as hopeless hack.
I wrote a long blog article this week about what I think the interview process should be, will be posting it soon. There are ways to fix this process.
Depends on the job.
I'm currently hiring and the live coding exercise is a must for the position because I want to see how the person problem solves while under stress.
Because stuff happens. My team works on mission critical systems that can have issues that we sometimes must resolve quickly which is very stressful.
If you can't handle the stress of me watching over your shoulder while you code then you can't handle the stress of getting a critical bug fixed immediately. Maybe that's not true for everyone but I haven't heard of a better filter.
Not every position is like this. But that's the big thing I think people miss. Different software positions require different skillets beyond just tech stacks. And the interview process should measure for the particular software engineer skills needed for that position.
I've also hired people who if you just reviewed their code after the exercise you'd think they were terrible. But I'm not testing whether they are good at coding exercises, the test is a tool to see their ability to problem solve under pressure and to see their level of experience with the particularly tech stack.
Those are two very different kinds of stress.
In one situation you're stressed because you feel like your every action and step is being judged. Every moment you spend floundering you feel is being docked against you, you can't help but worry about how the person breathing down your neck is expecting you to approach the problem.
In the other situation you are a part of a team, you have each others backs and trust each other to make sound judgement calls. You're working together for a common goal rather than judging each others every movement.
Your hiring method sounds like hazing, you're putting interviewees though unnecessary stress to see if they crack. Stress that isn't the same as they would feel on the job, at least I hope you don't also treat your employees the same way.
To echo your point, fixing a bug with someone you just met standing over your shoulder clock ticking, is nothing like fixing a bug at your regular place of work.
In an interview, if you don't perform the task well, you can fuck off back to the street where you came from. In your job, you get to ask colleagues, consult previous project code, refer to in-house or external documentation, and calmly analyse to figure it out under your own "in the zone" steam.
Remember, interviews go both ways.
This doesn't sound like something a new hire should be responsible for and more someone who would eventually, after proven themselves able to, participate in.
Interviews are already stressful; you don't need to add to that by putting candidates into even more stressful situations just for kicks.
To flip it around, would you be willing to let the candidate review the details on all technical employees exit interviews and then ask you about them? Maybe review some completed financial audits from previous years? so they can see how the company reacts under pressure?
It doesn't.
> I'm currently hiring and the live coding exercise is a must for the position
It's not.
> because I want to see how the person problem solves while under stress
Why, it's a totally bullshit proposition. The stress of not having memorized some algorithm is not the same kind of stress as a live system going down.
You do it this way because you aren't able to come up with anything better.
Otherwuse these exercises would just paint a broad stroke that handling stress is common & generalized, when in fact it's unique and case by case in most situations.
Sounds like a great place to work...
tl;dr: If you're a stress factory there's no way on earth I'd be willing to work for you in the first place.
Count me in that group. I have no desire to go through that BS. Seems like the bozos have taken over and ruined development work.
Given the negative response we've had in the past about take home assignments for senior devs (their time is pretty valuable, so I try spend the same time they do), I'm not sure what else to do to accommodate.
It sounds to me like you’ve done some careful thinking about this, and it doesn’t sound unreasonable. One of my beefs really gets down to the interview process turning into a one-way grilling, dehumanizing the candidate into a “code monkey.” It sounds to me like you are trying really hard to evaluate their technical skills, but in a way that is collaborative.
Bingo.
A hiring process that attempts to convert the multidimensionality of humans to a handful of numeric variables is essentially trying to hire the best drone out there, not the best human fit for the job.
I learn more about candidates from the types of questions they ask me, and their reply to open-ended questions I ask them, than from any technical grilling test.
So I would take writing software for fun as a plus, but would not take not writing software for fun as a minus.
(Being forced to use someone else's machine is a developer hell with so many dimensions.)
For instance, I nearly avoided very early carpal tunnel issues in graduate school by entirely relearning to touch-type on the Colemak keyboard layout. Sure it only takes a few minutes to switch to the layout on Linux and macOS these days, when I remember where the keyboard preferences are, but it still takes Administrator access and a software install in Windows.
Even "easy", that's still a cognitive load distracting from whatever the actual question was and starting into the actual problem. More importantly, at this point in my career, it's something that I'm going to see this as an immediate sign that employee ergonomics may not be a concern for the person giving such a test, which may speak to other aspects of the work environment.
This gives me a pretty deep insight into how they approach a problem, but also how they interact with others. There's no adversarial element, and it gives them the chance to see me screw up to give them the mental space not to sorry about stupid errors.
The one worry I have about this is that it's more vulnerable to my own biases than a more disinterested process would be, but if I'm consciously aware of how that can play out, I'm in a better place to counter them.
I don't care if they can write the code but I want to see some level of engagement with the problem and some understanding.
Not sure what happened inside my brain, but I simply froze and couldn't solve a fairly trivial coding problem.
That only thing that makes me feel better is that it was a one-time problem.
Turned out good enough, went onsite. Failed. Oh well.
Tried again a while later. Failed FB phone screen, got told Google wanted a second. That was the point I said fuck it and cancelled.
shrug
This is why advice for passing the Google/etc. interview usually boils down to: Quit your job for a month and do competitive coding, and you will pass with ease.
In the end, I scored a job with a FAANG company literally because I didn't want to work there. A friend warned me away before the interview for work-life reasons, but I decided to go ahead with the interview anyway as practice and I nailed it. There's no reason to stress if you don't want the job. Another friend who worked there later convinced me to take the offer.
What bothers me is: you're basically the same coder both before and after competitive coding practice. You're only more marketable with the same skills.
Yup. You keep keep measuring that thing..I don't think you're measuring what you think you're measuring.
I really hope that's true. Every solution I've heard so far is unrealistic at best, like using take home assignments, as if no one would cheat or complain about those.
I dont need to hit the book to write fizzbuzz, and i could probably code up dfs and bfs without prep but i dont walk around ready to retake my undergrad algoriths midterm. Especially when i really dont even know what the topic will be.
Im aware that the question i posed isnt terribly difficult and perhaps it does say somethingabout me that i need a bit more time to refresh my data structure to do this at a whiteboard.
But i do want to be clear that fizzbuzz is not representative of the 5+ hour whiteboard grilling ive gone through at nany of these interviews.
These folks have real data on lots of interviews, and have done some analysis on them:
http://blog.interviewing.io/you-cant-fix-diversity-in-tech-w...
The takeaway that's relevant to you is the section titled "Interview outcomes are kind of arbitrary". Specifically, things like:
As you can see, roughly 25% of interviewees are consistent in their performance, but the rest are all over the place. And over a third of people with a high mean (>=3) technical performance bombed at least one interview.
If you set up a typical tech-company interview process, and if you equate "didn't pass this" with "complete impostor and utterly incapable of coding in any capacity" (as most people with your approach do), well, congratulations. You're almost certainly rejecting a bunch of qualified people, simply because you're terrible at interviewing them and can't actually tell from your process whether or not they're qualified.
The most generous thing I can say is that it's not entirely your fault. You adopted ideas and approaches from people who seemed authoritative but turned out to be the actual impostors in this situation. But now you have a choice: you can reject that, now you know the problems, or you can double down. I suggest not doubling down.
The question is how can you test for experience in writing code that is maintainable, readable, 'instrumentable' and contributes to a better system. How often does a candidate refer and defer to a decent third party library instead of feeling they need to (re)implement a well known data structure or algorithm? What have they done to help automate things so they can free themselves and their peers from minutae (aka how willing are they to put more work on machines?). What more do they think they can do. How well do they mentor or are ready to be mentored? Given a sample project, how would they go about running the technical side of things. If they focus on code, you should try to motivate them to expand to other areas (without telling them what they are). How cognizant are they of the ecosystem; or does their vision end at coding + unit test.
Lots of intangibles that cannot be measured with a github profile or a white board interview but probably contribute more to picking out better candidates.
Like literally can't do a basic loop. I'm not looking for 'my solution', I'm looking for any solution.
> I think it's fair to say that if you get a senior Dev in that can't even write out an if-else statement checking for 3 and 5 you have a problem.
And yet, I can tell you from experience that there's someone out there who (at least was on the team that) wrote firmware for the space shuttle that literally can't if-else with modulus despite having a great looking resume. And they are far from alone.
Here's my FizzBuzz:
https://github.com/ubernostrum/interviewer-hell/blob/master/...
My code for "detect if a number is a perfect square", though, is O(sqrt(n)) time and O(1) space and uses no floating point operations. Which isn't quite Newton's method, but also saves me having to memorize an implementation of Newton's method and appropriately causes confusion. Here it is in Python, with obfuscation help from itertools:
https://github.com/ubernostrum/interviewer-hell/blob/master/...
And slightly less obfuscated C:
https://github.com/ubernostrum/interviewer-hell/blob/master/...
You need to ask for those, not just prior companies. My resume has a "References available upon request" at the bottom and I'd happily give them. Is that no longer practiced? It used to be a requirement.
Yeah, right. Personal references aren't worth squat because nobody gives references that are going to badmouth them.
The worst hire I've ever seen had a highly positive reference from a previous job.
Sounds like the interviewer didn't bother checking references.
College buddies will tell more lies than the candidate will.
Who cares if this person can't solve fizz-buzz on the whiteboard? That's probably not even a relevant skill to the job opening you're filling, or a skill they have ever needed to hone or practice. It's like determining a marathon runner's worth by timing their 100 yard dash.
If I ever need to do another coding interview again, I'm gonna troll the person asking the question by pulling out my phone, googling their question, and consulting the stack overflow post in the search results to solve the problem. And why not, this is how I (and the vast majority of people) will actually get the job done in a real situation.
A marathon runner doesn't need a stellar 100 yard dash time, but they should at least be able to jog 100 yards.
If you're working on the shuttle, those small tasks generally won't be surprises. They will have been planned out well in advance.
> Who cares if this person can't solve fizz-buzz on the whiteboard? That's probably not even a relevant skill to the job opening you're filling, or a skill they have ever needed to hone or practice.
Thing is, shuttle-style development is also probably not even a relevant skill to the job opening he's filling. Odds are, that job requires something more akin to FizzBuzz ('I want to do something. How do I do it?') than to shuttle-firmware.
Also, another thing to consider is how this kind of interview encounter negatively affects diversity and the experience of candidates from different cultural backgrounds. If we treat this form of interview a sort of hazing experience that doesn't really inform on their aptitude on the job, what kind of traits does this process _actually_ select for? Would be an interesting study.
Whenever I've suggested it a usually more junior guy has scoffed at how trivial it is... I'm yet to then have one supply a complete and flawless solution. These are from people I work with, respect and happily rely on to be able to code. Their recognition that the problem isn't so trivial for themselves is the only thing I've got out of it - I'm no closer to understanding if it's just a poor test or not.
There first time I came across the problem was as a candidate and I screwed up the upper bound of my for loop just because I was slightly thrown by starting from 1 instead of the almost obligatory zero. Naming things, caches and off by one errors.
My worst candidate experience was only a couple of years ago when they wanted me to reason about booking cinema tickets over the phone (like it's the 90s?!) I've never booked cinema tickets and I think they thought I was joking, but they went at me for at least half an hour on what users would need to supply other than location, film title, date and maybe time, to begin the process of booking tickets. Why they didn't just tell me what attribute they were after I'll never know - eventually they just wrapped it up.
Number of seats?
Agreed. The solution is to not work at a tech company, but instead work in a tech capacity at a real company.
- No silly games (before or after hiring).
- Better benefits (I'd rather have proper health coverage and a pension than a room full of toys and vaporware stock options).
- Immensely more job security (Henry Ford didn't have an "exit strategy").
- People respect you more as a professional with a needed skillset, and not a throwaway cog in a machine who can be replaced or offshored on a whim.
- And in many real companies, proper internal HR groups focused on making you a better person and a better employee, and not some farmed-out commission-based headhunter group that makes money on employee churn.
> In an ideal world, you could trust someone's resume.
That's not an ideal world. That's the real world. Meaning not SV, and its wannabes.
Some of the best career advice I've ever gleaned is to work on something that visibly makes money for the company. I get to do that every day at my tech company. I don't know that I could say that if I worked in a technical capacity in another industry.
Only by beancounters, and if you take it away and go back to paper, they'll quickly change their mind.
Good developers cost a lot no matter where they hang their hat.
9:00 - 4:30 daily hours, no nights, no weekends. Full benefits, good salary, good yearly bonus, quiet office, great people who were there to get a job done and then go home.
Had to wear a shirt and tie but that's not the worst thing in the world.
The only trouble is that finance is only in the NYC area. You can't be there if you are in the valley.
There used to be some in Chicago. Bank of America, Swiss Bank, First Chicago. Swiss Bank is now UBS and I don't know where they do their trading system development. First Chicago got absorbed by JP Morgan Chase. B of A might still do some trading development there, but I have no idea.
Back in the 90s Swiss Bank and First Chicago used NeXT for trading apps, and Swiss Bank had Symbolics machines here and there.
The skill I look for over all others is an understanding of the standard principles of software design, and principles of testing is a bonus. I want to see that the candidate is honestly interested in our language to the extent that they are familiar with the obvious pitfalls of the standard language features. Candidates who prepared by writing option pricers and passing leetcode challenges won't necessarily understand any of these, because they are simply writing functions.
I assume that anyone who got through a rigorous engineering undergrad has the mathematical preparation to perform as a quantitative developer. This won't be true for all roles, but it's true for all of the QD roles that I have contact with at BB.
If you are interested in a dev role in finance, and see requirements for basic financial knowledge, risk, etc. -- as long as the requirement isn't quite specific, such as experience pricing specific option types, pricing complex products, or with a specific product, I would just ignore it and apply. Many of these concepts are quite simple compared to what SEs are trained to model and implement, and the people on the team who need help know that.
I've seen GPU and C++ skills requested for a Pandas shop. Just apply.
I think hiring managers and HR think they will attract better candidates by adding these things, but it is counterproductive.
From what I can tell from salary surveys, we pay an entry-level dev better than many others. >150 guaranteed all-in first year, and better than that afterwards if you just do your job. Promotion path is good because modern skills are in dire need.
I left because it was my first job out of college and I wanted to see what else was out in the world. But it stays a fond memory of a solid job.
Yes, a lot of adults wear sweatshirts and hoodies now. However, if you wore a shirt and tie to a high school, they would make fun of you for dressing up like your dad. We have collectively agreed on a "grown-up" attire.
Yes.
Dressing for comfort, or to suit the physical demands of your job are valid motivations. I wore a shirt and tie at my first job (back in 1987), but within a few years, I didn't bother any more because no one else did. Nowadays, "business casual" is fine by me.
My initial reaction was to guffaw at the ridiculousness of such a statement. Expression through fashion and attire is pretty much the entire reason we dont all wear the same thing as each other every day. I feel the exact opposite as you: people who dont dress with the awareness that their appearance is a statement to the world they walk into are missing a massive opportunity to affect their day. How people treat you. How people respond to you. Etc.
But since I'm growing less cynical in my old age, I would care to learn more about your perspective if you care to share.
Which, of course, tech companies don't value at all. From what I've seen, they much prefer to hire a fresh grad over somebody with a long record of proven performance.
So, I agree that one should seek work in tech at a non-tech company.
In any case the same bank with Blockchain and big data lakes will also be working with mainframes and COBOL, so generalisations about technology aren't particularly useful when it comes to organisations that are so large.
Depends on the company. Where I am, IT is in charge of maintaining the computers and networks so that the software developers can develop software in and for other departments.
You're a carpenter. You nail boards together to serve the business interests of the company. Even if the boards are SQL queries or ReactJS or whatever.
Companies need tech. You can bring value by making everyone's jobs better with your technology - sand off those rough edges on everyone's workflows. Demonstrate your value.
I worked in a tech startup, where everyone did everything, and where doing everything was considered high-status: it meant you were competent at everything. Or perhaps the other way round: trying to avoid any computer-related task, for whatever reason, was perceived as an admission of incompetence, therefore low-status.
(I didn't agree with that philosophy. I am humble enough to admit that I am not an expert at something, e.g. setting up printers; and lazy enough that I want to avoid such work if I can.)
I also worked in a non-tech company with sufficiently large internal software development department, where as a programmer I focused on developing software; and if any part of the technical infrastructure stopped working, I just called the internal tech support.
In my current (non-IT/software dev) workplace, the CEO and the manager pride themselves on how all our staff can do everything. We're all sold on it with the line that "..all your future employers will be amazed at how much you can do!" Unfortunately, everything we produce is second rate junk as a result of this, as we are "all responsible for quality control," and the customers pay appropriately.
It's not that it doesn't work, clearly it does for some businesses, it's just that someone has to be responsible for making sure that the product is fit to be presented to the customer, and if everyone's running around doing everything, nobody's really doing anything.
There is always something you don't know, no matter how "senior full-stack" whatever you are. And when that happens, and the company culture does not allow you to admit it, the only solution is to use google, stack exchange, download a book or two, and perhaps ask a friend -- but neither of that makes you an expert overnight (and sometimes "overnight" is literally as much time as the agile process used in your company will give you; just because someone else, who actually is an expert in that stuff, could do that in a day).
The more I know about the things I specialize at, the more I am aware that you can't reach that depth of knowledge and experience by giving three smart questions to the search engine, and reading the three resulting articles. And by analogy, I assume the same is true about some of the things I don't specialize at.
But I have kids to feed, so sometimes I just bite my tongue and produce whatever is possible within given constraints.
This company culture also leads to playing games where everyone claims to believe that of course anyone can easily do anything -- but there are tasks that everyone is trying very hard to avoid: "It's not that I couldn't do that, I am just very busy now working on some other high-priority stuff that only I can do. Perhaps John would like to take this nice and simple task?" "Uhm, I am also in the middle of, uhm, something. Perhaps Joe could do this?" Joe doesn't have a plausible excuse ready (or is not present at the moment), so he is assigned the task. The meeting ends. If anything goes wrong, it's all Joe's fault.
In long term this reinforces the status ladder within the company, because the high-status guys are in the best position to refuse the tasks outside of their field of competence (by claiming they have more important stuff to do); the low-status guys get stuck with the tasks, produce mediocre results, and that is taken as a proof that they really deserved the low status. (If they are inexperienced 20-somethings, they will quite often buy it, and feel guilty about their incompetence. Then some moment later they change job, and realize it was all actually about the company culture.)
Laptops, printers, email, document storage and so on is part of the software development team's responsibility -- so we outsource all of it, and spend a little time on it roughly every 6 months for a new system or similar.
"Hey, I can't print". "Here's the helpdesk number for the people we pay for printers". (Some people were annoyed at first, but understood when they saw the developers' salaries compared to what we paid for the "boring" services.)
And, like, literally everyone since Adam Smith and his pins.
Assessing how good people are at skill A by trying to determine how good they are at ever-so-slightly-related skill B is what led google and the rest of the valley down the ridiculous problem solving interview quest of the aughts. Take your manhole covers and suck on them!
Abstraction is a skill, one that’s difficult to assess in an hour-long interview, but one which allows me to say “I’m great at C# but I’ll call someone when the printer is broken” - because there are only so many hours in the day.
Today? At 20... "what will I be doing for a living? Hm, I did some html and copy pasted js, I WILL BE A DEVELOPER and be RICH". Went to college, learned to develop. But I don't have any foundation, web and sql is all I know, and I am sleeping with Programming patterns book under my pillow and looking at latest programming hype (silently laughing at year of nosql databases and year of blockchain, now AI, and eagerly waiting for the next silver bullet that will solve ALL our problems :D)
IT is not a different field, but the level of today knowledge IS, also the attitude.
A friend of mine is having a successful software company and he said to me once: "If I get a guy with the attitude and background we had, I will hire him even if I wont be hiring at that moment. They became so rare, that it is worth having him on a bench, just to be there when you need him".
But joke aside, what is the issue, do you disagree?
The scales I work at are more about Big O than anything. If I make a mistake that is N^2 vs Log N then I wait a week for a job to finish verses 1 hour. That is an issue. If I add on 10 min because I inefficiently use swap space it doesn't matter. I need to pay attention to the higher abstraction rather than the minute details. Sure I can fix a computer but if I do that rather than stopping a job from doing puts across regions it will cost the company 12000 dollars for each hour I am removing malware.
Honestly, people who actually know sql enough to build a decent table design are super rare these days.
As far as everything else, you're pretty spot on.
Rather, we manage peoples computers, and the environment that allows them to function. The misconception that IT is only there to fix, is because we're invisible to you when everything works as expected.
0. https://www.forbes.com/sites/jwebb/2017/05/29/british-airway...
IT is by far more lucrative in the long run. It's just the structure of RSUs and stock options that gives start-ups the illusion of relatively larger comp gains.
We all know they are following business procedure, but there will always be some unconcious tension with departments that hinder a solution (I see marketing and legal butt heads all day).
In an ideal world, IT and Devs would be on the same page - after all who will be first contact when an end user encounters an issue?
The separation of development and support makes sense from a budgeting point of view, but operationally should be intertwined (IMO).
It's entirely reasonable for a carpenter not to want a job where they just nail pieces of wood together. Not really building anything, not really using skill. Just nailing a few boards together all day.
Carpenters build houses, woodworkers build furniture. IT people reinstall operating systems, fix printers and networking issues and software engineers write software. They have a different education, different job description and different pay scales. A woodworker wouldn't make a good carpenter nor a software engineer a good IT person or vice versa.
There is absolutely nothing wrong in being an IT person or a carpenter, they're reputable jobs that put food on the table. But if your trade is being a woodworker or a software engineer you should be slightly worried if you end up being put in a carpentry or IT. Or vice versa.
Disclaimer: I'm biased because I'm a software engineer and a woodworker. I'm good at my job but I can't fix networking issues or build houses.
It's actually at the smaller companies (like startups) where you'll be expected to do that. I worked at a company with <20 people total in the entire "IT department" (encompassing all computer-related things). Even though my job title was security analyst, I was in Active Directory, I was configuring Cisco gear, I had the on-call phone for the entire department, my phone was in the helpdesk rotation, I'd be dispatched to satellite locations to troubleshoot their wireless.
In my current job as a consultant, I've worked with startups where their engineers are answering support emails and replacing their own hard drives because they don't have a helpdesk or a desktop support team.
'exDM69, I'm not arguing with/against you, just piggybacking off your distinctions to help the conversation. If you're afraid that moving from a tech company to any other enterprise will have you putting on many different hats, it's probably not true. Unless you move to a tiny company, which is true for tech companies as well. Any real enterprise company will have dedicated IT support teams and you will be reprimanded for trying to perform your own IT maintenance since that's not your job.
IT runs the network, phones, issues and fixes laptops, maybe provides servers, often is in charge of information security. May own the financial and business intelligence technologies. Unless it's a very enlightened org, this is often the "cost center" team.
Digital talks to the world in a modern way--runs the websites, blogs, social media channels, paid digital ads, etc. Often housed within a larger communications and/or marketing division. Technically a cost center but the visibility makes up for it in the eyes of executives.
Product builds the technology that makes money. This would be like the e-commerce platform, ad network, mobile apps, etc. Executives value things that make money.
Often, all three groups need developers. The tools and roles may differ, but you can be a developer in a major company without feeling "stuck in IT."
The second (making a bottom-line impact) is probably a non-issue. If it's not a tech company, that means they are likely failing to use tech fully, in places that really could benefit from it. Automate stuff that needs it, and quantify the time saved as:
(time it took before - time it takes now) * number of times each person has to do it per week * number of people * what those people get paid / what you get paid = time T you saved in a week, in terms of your own hours.
Each time you do something, add its T value to a running total. Likely an hour here, an hour there. Eventually it hits 40 and you and the managers can clearly see that your employment is paying for itself. Keep going and you're making them money.
Yes, bad english can be detrimental: starts wars, malpractice, etc... We just take it for granted.
If code is treated like math, then we can talk about absolutes and good code.
But 90% of us talk about it in the language context, hence why we call it a programming language....
When you compare a non-tech company to a big tech company like Google, the benefits and comp arguments switch to the favor of the big tech company along with the respect and job security.
The only advantage to non-tech companies when comparing across big ones is the simpler interview.
The several hundred other auto makers in Detroit did, they went bankrupt or sold out for peanuts right before going bankrupt.
Google, Microsoft, Amazon, Netflix, Apple, Facebook, Cisco, Oracle, Intuit, Adobe, PayPal, nVidia, Intel, etc. don't have exit strategies either.
For the vast majority of businesses, you're either going bankrupt, or you're going to eventually sell as an exit. One in a million businesses turn into the next Ford scenario.
Ford is a public company. That's an exit. An exit is whatever makes your stock liquid, not the end of the company.
It depends on the company. MOST non tech companies imho treat all tech workers like IT, with the exception of embedded data/analytics people IF the company is even forward thinking enough to have that.
You compared the best examples of corporate with the worst examples of tech.
We do, it's called the probationary period. 30 or 60 days, or even a bit more. If the person isn't working out, or is not a good "fit" you can say goodbye. And it goes both ways. Candidates might find that the actual work or work environment is not what was portrayed in the recruitment process. If they don't like your company, they can leave.
There are times it makes sense to do the high-risk ones: devs that are unproven, without great academic and work backgrounds, looking to get a foot in the door. Companies that are not well funded looking to find diamonds in the rough, since they can't attract people who already have proven themselves. I've been that dev myself, based just on a strong sense of how the CEO and myself were on the same page based on our conversations.
But it's definitely not a risk I'd take again from where I am now. At a more established place with people I know who vouch for the folks involved, there's a much lower chance of a new FTE getting canned within a few months for anything short of egregious behavior.
And it's enough for someone with good social skills to get a great recommendation without skills.
But this tends to be rare in practice, of course, because the pool of potential people being hired is very large.
Also, in the United States, you could fire someone easily if they were really incompetent. Considering how much time and money people spend per candidate to interview, it's probably much cheaper and more efficient to hire faster and fire faster. Some companies are spending tens of thousands of dollars per candidate they interview!
Hiring is a skill, and an overly long or drawn out process is a sign of bad hiring, not good hiring. The second part of the equation is to train people up and mentor them. Hiring is not enough. The second big part of good management is to actually get the most out of people and to help them learn and grow.
Now if it were only the tech industry that had long, ridiculous hiring processes, maybe we could say that it is necessary, but many companies and many fields have similar processes. I believe that many people have mistaken length of interview process and time spent with rigor.
I'm not hiring people who can talk and not code. I'm hiring people who can code (and, ideally, talk). The only way to verify that they can code is to witness it happening.
Reality is otherwise. Lots of paperwork involved (for both hiring and firing). And leads to a not-too-great work culture.
There is very little paperwork required to fire someone.
If the employee is still on contract (ala contract-to-perm) just collect their badge and be done with it.
If they are a W-2 employee with benefits, you hand them a few benefits notice forms and send them on their way.
If they apply for unemployment, the company will get a notice that the former employee has applied for benefits.
Plus there's the added risk of hiring "duds" that are also minorities. And letting too many of them go will make it look like formal discrimination. Which is a much, much more expensive mess.
And then there's this issue of finding the duds. Easy for the people on the ground to spot, but less so for management and HR, which will require some form of reasonably accurate performance evaluations.
The Glassdoor reviews on Netflix suggest otherwise.
1. Expectation of long work hours (Frequent and short deadlines due to agile)
2. Micromanagment being the standard (agile)
3. Terrible environments (open desk/office)
4. Removal of vacation time (unlimited PTO means you have no vacation days)
2 week sprint deadlines are entirely too short. say there's a larger project, it takes a lot of overhead to split everything up into neat 2-week releasable pieces. After the slicing and dicing, the big picture gets lost and garbled, adding more stress to developers.
Third, and most importantly, it offloads responsibility and ownership from management (especially upper management) onto lower level employees. Why should managers do their job if agile teams are "self organizing" and "self managing"?
On top of this, I'm not convinced product or business types are better able to design a good product than most developers are.
That's just not universally true at all. Using my current employer as an example, they have been very good about encouraging employees to take advantage of unlimited PTO.
Managers take PTO and encourage people that haven't taken it in a while to do so. Even the CEO goes in front of the company at all hands every once in a while and tells people "I just came back from a week vacation, you guys should too, remember to take time to recharge". It's something you have to get right in the company culture early on, granted.
I personally am so spoiled by this that the thought of having a set number of vacation days gives me anxiety. I don't want to count vacation days. It's not that I take a ton of vacation, I'm not a big traveler. But I like knowing that I can go to my manager and say "I'm taking friday off cause I'm going on a road trip this weekend" and he can just say "cool, just put it on the calendar" without worrying about counting days.
That said, I know this is really dependent on the company and have been in the opposite position at a company where PTO wasn't counted but was culturally discouraged, so I get where people are coming from. But I wouldn't just flat out dismiss the idea like so many on HN do.
> I personally am so spoiled by this
I had the standard six weeks of vacation per year at my first real job. How many weeks of vacation do you use per year?
I know, I was trying to add some outside perspective. Where I'm from, 5 weeks is the legal minimum, and everyone has the legal right to get 4 consecutive weeks off in the summer. Consequently, since everyone does this, and expects this, companies adjust for it, and most importantly: No-one bitches about it. There's no masochist culture around not taking vacation.
The problem with "unlimited" vacation is that it's of course not unlimited. There is a limit, it's just not written down on paper, it's not part of your employment contract. It's arbitrary. At one point, either your boss or your boss's boss is going to say no.
And if you get told no, you have no legal recourse! If you get fired for taking "too much", you can't sue for wrongful termination!
Fixed vacation time protects you as an employee. Why doesn't your company just give everyone five weeks of vacation and have that written down in the employment contract?
It's exactly like bonuses vs. salary. A bonus can always be withdrawn for any reason or no reason, but it's very difficult to lower someone's salary.
If it's written, it means that you have the minimum legal number of days in your jurisdiction and you are being screwed severely.
Passing off someone else's open source as yours would be a hard lie to keep under wraps, an easy lie to uncover and I've never heard of anybody doing it to score a job.
Have you ever heard of this happening? Even once?
I now have two steps before an in-office interview: an online programming test, followed by a screen-share coding test. When I just did the screen-share test, I can't tell you how many people claimed to be java programmers and their IDE was eclipse, and then they didn't know how to start coding or run a program. "I know junit", ok, write a junit test and run it in eclipse: deer in headlights. Such a huge waste of my time. So now we do an online programming test, and many applicants fail to write code that compiles, let alone passes the test.
There are good applicants out there. It seems like they are vastly outnumbered by frauds. I have no other way to describe these people that cannot program, and just show up with a bullshit resume hoping for a paycheck. Before I was responsible for hiring, a man was hired on our team who could not program, despite a resume showing years of java work. Now he gets to go to the next place with our company on his resume. These frauds are hindering us finding the genuinely talented.
Then you're not doing a very good walkthrough. I can generally spot someone who didn't write the code they are presenting almost immediately. Occasionally, someone might fool you, but it's pretty rare.
As one of my interviewers said many moons ago: "Yes, his claims and credentials sound overblown. But after reviewing his stuff and interviewing him, he either is exactly what he claims, or he's the best damn liar I've ever seen. Either way we should hire him--the only question is whether for engineering or marketing." :)
The best interview I had from a technical perspective was when I was asked to bring in my laptop with some of my code. , talk a bit about it and add a quick feature. No pressure as I knew the code and the framework. I felt like I could show off what I was good at. It would have likely caught out impostors as well.
This is true. Often the types of things that folks look for though are nuances and style that come through with what someone writes. Do they have comments in their code? What standards are they following? Do they have test cases? How well do they handle exceptions? Etc.
The structure and accoutrement that comes along with code / work product can be quite telling.
The above being said I'd say what you did in the "best interview" was very appropriate as well. What I've generally seen out there is a "homework assignment" is often a one size fits all as an attempt to eliminate qualitative bias ...
Working for nothing __is__ obnoxious.
Not to belittle anyone but...a plumber is licensed. An electrician is licensed. Even a hair stylist is licensed. Perhaps licensing a programmer / developer / engineer would be over doing it. (Read: Yes, it would be. But the examples do help.) That said, can't there be a formal certification(s)?
If the time from these "take home test" was invested in a more universal certification process, wouldn't that be to everyone's benefit? A recruiter with value add, if you will
The employee gets a clear picture (read: context) of where his/her skill set stands, or doesn't.
The employers get to mitigate risk and fear.
All parties gets to reduce friction and time.
Clearly, no one is happy with the status quo. That's a problem. It's also an opportunity. Hint. Hint. ;)
Shitty plumbers are licensed. Shitty electricians are licensed. Shitty hair stylists are licensed. A license says nothing about current competence. It says the person demonstrated some minimal level of competence at one point, fills out their paperwork every year, and hasn't fucked up royally enough yet to have it revoked.
Every time this subject comes up someone inevitably points to licensure as some sort of panacea. Licensure does not replace skills or competency assessments.
https://en.wikipedia.org/wiki/Dietitian#United_States
--edit: I think you might be talking about a nutritionist (https://en.wikipedia.org/wiki/Nutritionist) which is not regulated in any way and is not a medical job. These are the people who will talk to you about essential oils and homeopathy and Herbalife.
It's also why I pivoted to certification.
There is no panacea, nor is the status quo sustainable. So where's the disruption? Where's the innovation?
Because a license/certification test is the exact definition of "skills/competency assessment". If the licensing test doesn't meet those needs, that's not the fault of licenses in general, that's a fault in the test and can be fixed.
That is not at all what I said. I was addressing how licensure does very little to prove competence for the three trades specified in the post.
> Couldn't it just be that a "programming license" has regular testing and re-certification?
How many licenses require recertification? Why do you think a programming one would? Continuing education requirements do not count for this, as they are mere completion measures.
> Because a license/certification test is the exact definition of "skills/competency assessment".
Yes, and this test needs to be passed once. The longer ago it was, the less it says about the person now.
Because it doesn’t exist yet, and my hypothetical is just as good as yours.
That's 4+ years of study right there, that we did to prove that we could do the job, but no, fuck that, some of those people freeze up during an interview and a handful of those people we brought couldn't demonstrate their programming skill, so we decided we can't trust that for ANYONE.
Sorry, go invent some new way to certify yourself. How about 2-4 years of intense study and a new test, like a bar exam that lawyers take! Or just get your PhD! Just study for half your life, go waaaaay into debt, and take the exact same salaries we are offering now, because we don't reeeaaaaally want to pay you as much as we're paying now, so we're sure as hell not going to offer to pay more for that extra effort.
If you thought a day long test was too annoying, then your only alternative is spend even more time and get something we're just going to ignore again the instant we make a bad hire with someone with that license!
(For the record, if I could take a single certification test and skip all this interview bullshit for the rest of my career, I would have done that years ago. I hate going through the technical interview gauntlet, and I'm one of the good programmers, at least once I'm on the job).
How often do you hire some one based on certificates?
Unfortunately there are about 1000 competing bodies trying to be this association. It seems that the tech crowd would rather in-fight than work together to be treated seriously.
GitHub and code samples can be useful, but not every candidate will have shared code on GitHub (in fact, most don’t) and retaining copies of code they’ve written for a previous employer is a major red flag.
The biggest issue with using prior experience is that it’s hard to compare the ability of programmers who work in different languages and different domains, it can be like comparing apples and oranges.
I believe it's actually easier. If everyone is given part of the whole and 4 out of 5 people get theirs done properly and 1 is flailing, everyone, and I mean everyone on that team knows it.
Actually "open source contributions" isn't just about having some code dumps on GitHub.
First of all you've got the stream of commits, which is very gradual and shows you how the project evolved and since when. And you also have the issues, the pull requests, the interactions with other people. Those show you how the candidate communicated and collaborated with others.
You know if the code is his own or not because it's usually all there, everything that's needed ;-)
The industry's hiring practices aren't broken. What is broken is our ego.
How do hospitals make sure that their surgeons can do surgery without having them do surgery? Do they have some magic eightball?
However, CS is close to something else: Economics. People spend 4+ years learning about economics, but don't know shit about how it really works. They found a solution for that: business schools.
CS at a University versus CS at a technical college.
Or what about non-technical middle management? Or HR?
The solution? Sure, come in, do a shift. See how you like the kitchen and the environment. We'll just give you some cash for your time.
Interestingly this goes away at a certain level of seniority, but I always liked it as a simple way to get a feel for a potential hire.
Check out interviews and jobs section on pilot forum for comparison with our field: https://www.pprune.org/interviews-jobs-sponsorship-104/
I've also seen people that can ostensibly get the job done, but will write code that is so byzantine, so incredibly cryptic and opaque that no one else can understand it. I'm not talking about code that is incredibly clever, although that can be a problem, but code that is simply orders of magnitude more complex than it needs to be. From management's point of view, they've accomplished the task, at least in the short term, but to the people who have to deal with their code, it can require more work to make fixes or improvements that it would take to rewrite it all from scratch.
The real reason that it's so hard to hire a good developer, IMO, is that there are very few good developers, but a huge number of bad developers that have managed to convince their bosses that they are good, or at least competent, because there are also a not a lot of good development managers.
Can we really say that software development, as an industry, is really any better than it was back in the days of "The Mythical Man-Month"? Sure, the tools are lot more sophisticated and powerful, and the hardware has exceeded any reasonable expectations, and even imagination in some cases, but I don't think our ability to develop software has really improved all that much.
If you are going to devote 40~50 hours a week for a company thats going to pay you 6 figures and you are not willing to work a day or two, you have your priorities crooked.
And yes, just having a resume is not enough to figure out the good from the bad ones. If we knew that perfectly , there would not be any need for any interviews of any kind.
I mean good lord. People are asking for an objective/verifiable measure of a skill that's entirely in the domain of your supposed education and/or work experience. How much more straightforward can things get? Would you prefer that it be based on ... who you know in the company? How much you can bullshit your way through a soft skills quiz? How much buddy-buddy rapport you can establish with your interviewer?
These are objective measures on a set of borderline "well known" problems, ... I can't imagine a simpler thing to prepare for. There is no guessing. The amount of hoping someone just "likes" you is at a bare minimum. The impact of resume fluffing is minimized.
I literally can't think of anything more fair or objective.
I can't even fathom people that aren't willing to make these miniscule one-time efforts where there are literally buckets of money waiting on the other side of the gate. Money that most non-tech people will never ever dream of obtaining.
I'll take all my downvotes now, but I'm willing to karmically die on this cross. :P
I'm absolutely willing to lose a couple days for a job interview, if I have a decent chance of getting that job.
If a company is asking me to complete an 8 hour test and they have already received 100 qualified applications that have also completed that test, I should know how many people have successfully completed that test.
Giving a free 1/2 week of work for a job you have a > 10% chance of getting is very different than being asked for that kind of work for a 1% chance.
How about a 4-5 hours coding rush in the office, where you have to add a few functions or features to an existing codebase?
I don't follow what you're implying here, sorry?
> How about a 4-5 hours coding rush in the office, where you have to add a few functions or features to an existing codebase?
I work at a company where we actively keep a small set of tasks groomed for new hires. It's extremely difficult to keep a set of meaningful actually-in-the-codebase problems groomed at all times and totally infeasible that the person will spend less than a day coming up to speed on the infrastructure etc. The overhead of doing that is really prohibitive from an interview perspective.
I do think that for the most part, the coding problems need to be synthetic, then, not an active need in your codebase. I am definitely a believer in longer more open-ended coding challenges than 45 minute white board jams though.
I think in general asking you to do a coding assignment in the office is better than asking you to do homework because there is a cost (in terms of time invested) to the company. For homework, a company can ask dozens of applicants to invest hours of their time at minimum cost to the company, even when some of bthose applicants had no chance to begin with. Here, they're forced to be choosier.
Imagine nurses or docs getting asked to take vitals for an interview.
There are companies who will only hire you if you got a degree and relevant experience. The big tech companies never mention it but most of the workforce had a long education.
Requiring a master's for a dev position is just a joke and I wouldn't take a company seriously that was asking that.
Above not true for roles like PM, where for better or worse an MBA carries weight.
I've seen it in almost every place I worked for in London. The floor is either only people with degrees or only people with no degrees.
The degree is never announced as a requirement, it's rather selected automatically during the interview. For instance in programming, some questions about state machines and graph traversal will do a wonderful filter.
Going back to the medical analogy, just because someone has a medical degree doesn't mean they're a good surgeon. They'd need additional certification to prove their surgery skills.
It would never happen real world, but "there are so many frauds out there, this is what we came up with. You should now how to take vitals."
Anyone can call themselves a software engineer, even if no one has ever tested their ability to reverse a linked list, or solve fizz buzz, or even write 'hello world' in a language of their choice.
This kind of comment is exactly the kind of thing that makes some people think that software developers have no idea just how good they have it.
That being said, I can't think of anything more viable than actually testing the programmer in question with code. I would personally give them a computer and have them do it right there on the spot in the interview but that is just me.
If this were an actual employee‘s market, the friend might not be so underemployed.
Not if you're geographically stuck and your skills are not in demand locally.
Your 1:3 odds will be different depending on your target. Maybe the commenter to whom you replied is trying to get jobs paying 50% more than their current job, but for each of these jobs there's only a 3% chance of getting it.
That was annoying, but considering I was unemployed and job searching full-time, it was manageable. If it becomes more common, as seems to be happening, it'll absolutely become an issue for anyone who has to do more than a couple of job applications, which probably includes most people trying to move up in position or relocate.
My sentiment in my original reply is that I hear a lot of complaining about "i was asked to code binary search" interviews, which seems to come with an implicit but unstated "...and i failed to code binary search, therefore it is bullshit" statement attached to it. Somehow I feel like if it ended with "and I coded binary search because I can generally code on-demand or because I specifically practiced it, and I got the job!" -- like there would be less complaining happening...
I mean, people don't wanna do whiteboard coding ("it's not realistic"), they don't wanna do take-home coding ("it's too much work") ... are people willing to do ANYTHING to get a dev job?
I've seen a situation where the person literally refused to do an interview, just stated their desired comp, take it or leave it. Are you fucking kidding me?
The statistics on this speak a far different truth - one where most of the time, you're going to get rejected no matter how good you are, because you're competing with hundreds of other candidates for one position.
But I hadn't talked to anyone in the company prior to them giving me the homework. So they are impressed and move me on to the next stage, I talk with them and immediately find out that 60+ hour weeks are the normal for them several months out of the year. So my time on the coding exercise was wasted, a simple phone screening where they gave me details of the position would've saved me from that and I now insist on a quick chat with the hiring manager to "find out if I'm right for the role" prior to completing a non-trivial exercise.
People are just arguing where that line should be drawn. Whether it's hours, days, or months of verification. Some careers do require months of unpaid internships in order to get jobs. And it's not because it takes months to verify a person's ability, but because companies will do anything they can to pay people as little as possible for their work.
I've worked on many great teams whose interview process was little more than asking intelligent, pointed questions about previous work experience. Of course, this puts the onus on the interviewer to know what they are doing.
I personally keep a list of elementary questions about various technologies for interviewing. So when someone says they've worked with ElasticSearch or Python, I ask them how to do a few rudimentary tasks in a few of them and can pretty quickly tell how much they've been fibbing.
I've had companies both accept and reject my hiring recommendations, and every person that's been hired despite my objection has been poor quality. So I know at least anecdotally that it's a viable strategy and it only takes ~30 minutes per person.
As is quite normal in this industry, both of us occasionally have to put in large amounts of unpaid overtime in order to make deadlines. That doesn't translate into more money or career fulfilment for us, because the city we live in has a very low ceiling for this sort of thing. Not everyone lives in LA, SFA, or NYC.
Happy to discuss this, though (and i assure without any downmodding!) I disagree with you in part. One issue is the amount of time required early in the process. A whole day of whiteboard exercisis plus prep time is a lot to ask wheon only, say, 4% of the people who go through it are offered a position. Keep in mind that unlike the bar exam or med boards, devs go through this every time they apply. This pushes a tremendous inefficiency out onto the field and almost certainly deters people fromapplyingfor jobs at all.
Which brings me to my next point- who is doing the majority of the complaining? Tech companies are constantly bemoaning the shortage of good developers. Im ok with a company not hiring me because i wont spend 40 hours on their unpaid homework assignment or exam. Seems they need to accept that just as i may not be hired, they may not do any hiring, and their own processes are the reason.
Amazon, Facebook, Google? Those are hard salaries to get in my experience. I'm reasonably well respected in my field; attending conferences, speaking, leading meetups, and tirelessly researching computer science in my free time. Can't get a job at those 3, but going to try and take the hedge fund route I guess (I'm in NYC).
Why do you say you /can't/ get a job at those companies?
Risking downvotes is having to remind you to read TFA, where they address the "objetive measure". Hint: they're not.
For instance, one company asked me to do this extract/load to Spark and analyze data thing. I did it because their e-mail said that they would reply to each individual with feedback. Guess what? 3 weeks later I get a canned response!
Another job assignment took me 4 days but landed me an interview that went really well and gave me a chance to talk about my code one-on-one with a real profession (I consider myself a newish developer). After the second interview, I'm told I almost definitely got the job and that the position pays a whopping $15/hour in NYC.
Right now if you offered me coding work for $15/hour, I would probably take it and be quite happy for a while...but if you put me through the wringer like that, make me explain all 500 lines of codes for 3 1/2 hours after I spent the entire work-week bootstrapping your next project, I expect you to at least negotiate around the minimum salary I wrote about on my my resume! And if you're going to flat-out reject me, that's fine but at least offer me feedback on my application or code!
I really enjoyed doing the coding exercises, it forced me to consider things I wouldn't think about when coding for myself. But I have to say, the whole thing is making me a little wary of companies that do this. I'm not saying I should have a job but a little reciprocity and honesty goes a long way.
PS: I'm a contrarian so I upvoted your rant.
I'm guessing Etsy didn't offer you $15/hour, right? :)
Seriously? You did all of that? For a take-home interview problem?? Oy. This is why companies get away with this crap...
I'm going to say something harsh, but you really need to hear it: you failed at your number one responsibilty. It doesn't matter that you enjoyed the project or got the job. You need to maintain some level of professional self-respect in these situations. Professionals don't give away their time for free.
If Etsy asked you to do this, they were being abusive. If you did it because you enjoyed it, you signaled to them that you're desperate for the job, not very busy with other interviews, or willing to do work for very little in return. Or some combination thereof. None of these are great signals to send when you want to be treated with respect, and catastrophic signals to send when it's time to negotiate salary. Your time is not worthless.
To put things into perspective, I made $30/hr 23 YEARS AGO as a summer intern (coding) in my hometown city's IT department. In an "inland" state. Not even at a monied startup. This was before the dotcom boom, and it was a "government job", which never pay as well as private work.
Anyways, check your inbox I'll send you some more details. I'd love to hear your take.
Why is an employer asking for "proof of capabilities" tantamount to abuse? Seems a little hyperbolic, yeah?
If you think "don't work for free" requires some sort of logical/evidentiary defense, I'm not sure how to help you.
I almost took it just to get my foot in the door but decided I could do better with 3 months of polishing up my portfolio.
What's really simulated by going off and building something on your own start to finish? You miss out on all the collaborative skills required to ensure you're writing the right code.
If you only care about hiring programming, motherfucker (google it) then by all means.
My rant is more about an otherwise normal interview, which has in-person coding, and possibly longer at-home demonstrations of work.
2. You mention the money. Tell me, what is the exact dollar amount that one has to make before they have to stop complaining about this?
The short of it is that it's a candidate-unfriendly measure, and in my experience has been correlated with having other red flags.
There are lots of professions that make just as much or more as your average software developer. Accountants, sales people, bankers, lawyers, dentists, managers ... the list goes on. Software developers are middle class (or at least what's left of it).
Is the homework a strong enough signal? What if someone helped the candidate? What if they won't get along with the team or the culture? How close is it to the real work the candidate would be doing? I wouldn't base a hiring decision on homework. So this is basically just a screen, a very expensive (to the candidate) screen. Who are you screening? Anyone good enough that doesn't need to invest a week to get a job, that's for sure.
If you feel homework is a good indicator and you ask someone to do that for you, pay them. $200/hr sounds like a good starting price.
I am pushing 100. Man, if one of the first 10 I did panned out that would have been awesome!
Requiring homework is a great way to deter the kind of people companies should be most interested in hiring: those who are not actively looking for a new job.
I find this trend especially surprising considering that the U.S. has at-will employment where you can fire anyone anytime (at least that is my understanding, I'm from Germany). Why not get people at a desk quickly and evaluate them on the job?
I'd much rather do that extra work during the hiring process than have to deal with a bad hire later.
This would be additional income complicating tax, and many companies restrict employees from taking on outside paid work without approval.
No matter how you slice it, these multi-hour homeworks are a stupid idea. And the real reason for them is to discourage candidates to justify getting indentured labour instead. Or to discriminate against older workers with more existing time commitments.
It seems wrong to reject a good idea because other bad idea will interfere with it. Maybe I'm fortunate; I take extra irregular work in the sum of about 30 days a year, and at the end of the year HMRC and I settle up very simply and easily. I couldn't imagine taking a contract that said I couldn't do any other work; it's by no means uncommon in the UK for people to have extra part-time work that fits around their main job. I think we shouldn't reject a good idea - paying people if you want them to do multi-day interviews - just because other bad ideas (tax systems that can't handle earning extra cash, employers who view their staff as property) will make it awkward for some people.
It’s usually about avoiding conflicts of interest - you can generally get the box ticked easily enough to do something unrelated to your job with them. But “doing an audition at a competitor” is unlikely to be signed off!
But even so - these homework assignments are stupid, and discriminatory on factors unrelated to job performance, and no other industry does it. In fact they yield worse candidates because quality candidates don’t tolerate this kind of nonsense.
And how would this work - you take vacation to do your trial period? You quit to do a trial period knowing it’s still an interview not even an offer?
We want people who are desperate, people who will give up their precious free time to do our bidding, people who don't know when they're being asked to do something plainly ridiculous. We want people who will give up their vacation to try to please us. These are the people who will embrace poor working conditions and tie themselves to us through Stockholm syndrome.
IMHO, hiring based on homework is no guarantee that you won't end up with a bad hire anyway. It is just another imperfect data point to base your decision on, but one that comes at the cost of massively reducing your number of candidates.
There's not much I can do, it's so common a practice nowadays (literally all of my team members are in a similar construction and we are working on fixed projects from nine to six) that with my next workplace I will do the same.
"Why aren't there 100x more people than I actually need (because we only hire top 1%, right?) with 100 years of working experience with this technology that I just dreamt last night that want to work for free?"
The industry requirement to prove that you can produce software is not unreasonable, coding is a craft and selecting developers is like selecting painters: ability to bullshit your way through an interview is orthogonal to the skill.
What developers have is some "game", which makes them a "player". As soon as they lose their game, they cease to be players. Their relation to the game is exactly their value and nothing less. Tech employers are higher level players who know that the economy takes no excuses, and they will trim the fat, so to speak, of anyone who cannot contribute to their gamesmanship.
Generally, in most industries, seniority is respected in and of itself. But in software, you see disgraced older people who didn't play the game right, and you wonder if they are to decline away out of sight. That suggests this is not grace granted by another party. Does that look more like a privilege granted (by whom? charitable businesses?), or a loser in a cold and harsh game?
I see more question marks over the futures of software people than I see in other industries where other peers have gone. In other industries, I feel there has been more of a multi-factored consideration for the generational passing of the torch. In software, I feel everyone is always in a sink or swim test, and I think older people sink a little more.
I agree we have a problem with age discrimination, but I can't conclude if our problem is any better or worse than the rest of the work force.
What does it mean if I say you are privileged? When one looks further at the definitions listed for privilege, one sees examples discussing kings and royalty for birthrights. If we're discussing whether people are unworthy, then we are on the same page.
Put another way, privilege is often used when some people tend to experience a fairer world than others. But expecting a fair world is how it should be! That's not "undue grace", as everyone should experience a fair world.
(Just so we're clear: this is a full-on semantic discussion, and I'm okay with that.)
That's quite a narrow definition. A privilege is any kind of advantage enjoyed by virtue of belonging to a group as opposed to personal merit. It's not fundamentally imoral and may well be temporary. Certainly no one has to grant someone the privilege of beauty or intelligence.
In relation to other professions , programmers have the strong privilege of being highly in demand in the labour market. This is clearly a temporary situation, and they should absolutely make the most of their "game" towards the tech employers.
What I was underlining is that the tech job market is ballanced, and certainly more favorable towards labour than almost any other.
I saw focus on meritoriousness, undue worth, or whatever is evoked under the sense of "birthright". I'm arguing that it's gamesmanship at play, and not metaphorical birthright, and hence I speak of some of the losers of the game, as well as the higher level players who interface with the economy directly.
The picture gets worse when you look at typical interview panels at hot SV companies: median-age 26, went to Stanford/MIT, interned at AmaGooFace, base their hiring process on established "rigorous" stack ranking of candidates based on their performance on fixed algo/whiteboard questions. While I can generally do pretty well at these, variance is high and it does nothing to differentiate me from smart but inexperienced people.
I think it's ridiculous that SV treats programmers like sports players with a limited shelf life. Code bases would probably be a lot better, and workplaces more pleasant to work in if experience were more valued. However I hold no illusions about the way VCs work: they require a steady influx of impressionable young blood to exploit with visions "changing the world", so it makes sense that older jaded devs don't fit into that picture. I do think there is an arbitrage opportunity for older talent though.
If I went to google or FB or whatever, it would likely be a nightmare of constant sink or swim projects and unrealistic expectations.
You’ve never worked in banking I see. There’s a 3-5 year grind of doing nothing but updating Excel sheets 80-100 hours/week at entry level. Pay is decent for the 1-in-10 that make it through this stage (i.e to Associate 3 and up) but the hours are still long and the work dull.
And there’s nothing at all straightforward about guessing which language or “framework” will be fashionable next...
They wanted me to build an app, with backend and frontend, and design (the design had to be "beautiful"), test it, all in under a weekend while I still was working fulltime at some other job, and I told them, they knew.
It was a Deutsch company. The name is GuteFre.The worst take home test ever, of course I didn't do it.
Germany also says it suffers from a lack of developers, but in my opinion what they don't say is that they need more developers to treat poorly and with a low salary.
Developer position, independent contractor, 5+ years experience Erlang, COBOL, Salesforce, Active Directory, IBM Mainframe, and Java required. Starting salary: $40,000.
And they're whining that there is a great developer shortage in America but they did get some offers from overseas that claim to have those skills, if they could only get a H1B...
Give an employer absolute power over an immigrant worker with no connections, support network, or other job options and their residency status on the line--what could possibly go wrong?
There is also an filtering effect on job sites where good jobs get snapped up right away and disappear from the site, while delusional prospects concentrate over time, causing people to think that those sites have nothing but garbage offers.
Where are all the good employees? They have jobs.
Where are all the good jobs? They have employees.
Every time I've worked at a small company that had obvious perks (using a cool language, building a cool product), we had a bunch of applicants to pick from. We could be picky.
But the few times I worked at a company on soul-sucking work like internal legacy tools in VBScript for a non-tech company, they were the ones complaining about the great dev shortage.
They paid well but you also had to jump through hoops to see what it was which is too typical in our work culture. So if the job description doesn't compel someone to apply, GG.
Mid-level position, asking for 5 years of related experience? 85% of the resumes have no related experience. Fresh out of school, that is. The virtual pile gets much shorter when minimal criteria are applied.
If I want somebody with 5 years of experience, it's because I need someone who has already got familiarity with the tools and can apply good judgement.
I have had both positions open simultaneously on occasion, and discovered the same people applying for both. Some of them were even qualified for the junior position.
I think our contention centers mainly around this word. When I see the word related I envision most standard job offers that have a laundry list of 10+ technologies, not all of them particularly related.
Software engineering is such a wide field with such a huge array of available technology options that if you're limiting to 5+ years in a specific stack you're already massively narrowing down your field to a small percentage of the available workforce. If you aren't offering significant advantages to offset that huge initial filter you're not going to get many candidates you find acceptable.
At the company I work at we hire for "general software engineering ability". You can pick whatever language or tool you want to get through the interview, we don't care. Most strong candidates will ramp up on whatever specific stack way more quickly than you expect.
If I ask for five years of related experience, I mean that if my list includes an object-oriented language with a well-known framework, I expect to see someone who has worked with an object-oriented language with a well-known framework and has perhaps dabbled in the particular one I mentioned.
Instead I get many resumes from people who have never used an object-oriented language to contribute to any software project that wasn't assigned by their professor. They haven't got five years of experience, period.
Does that help?
The worst I could get was a no :)
And you may even say you want "rock stars", completely forgetting that actual rock stars don't need 5 years of experience to do an exemplary job with a new tech stack.
Most job postings are complete BS, cut through that and the quality of candidates will likely improve.
Because then the other developers would not be able to haze the candidates. Seriously, there's way to much ego involved in development these days.
I went to a large state school for undergrad and tutored people regularly. By senior year, even the bottom performing students wouldn't have had a problem with fizzbuzz or similar screening problems. We regularly had coding problems much harder than that on tests.
Unless there were other factors involved like maybe the pressure caused by the weirdly adversarial hazing process that the tech interview has become.
It's been 7 years and he has never worked as a developer.
If you mean can't program fizzbuzz or simliar, how did he make it through tests? Did he cheat? In most classes I had, tests were around 60% of your grade.
But halfway through I got stuck. Something wasn’t clicking. This is stupid. I could do this in my sleep what’s going on? How much time do I have left?
So I did the only thing I could do. I wrote more tests, figured it out and got the job. Became a lead.
Conversely I’ve worked with many people who can handle trivial problem after trivial problem all day long without breaking a sweat. Give them data that isn’t arranged the way they need it though, and they crumble. Convoluted code full of redundant decisions or data rearranged in an inconsistent (or even data loss) fashion. Always had to be rewritten because we had five more features to add in the same place and you can’t build on sand.
Until we can get useful credentialing in place, my vote is still with white-boarding, with a preliminary remote coding interview over online tools.
I've been on an interviewing spree in January and February, totaling around 20 companies. There were maybe 3 or 4 with realistic and respectful hiring process.
In the end I was left with just one company I actually liked.
I end up having to hire new devs for my offshore team in India every 4-6 months or so, and I just haven't found a good way to ensure candidates are any good.
This is common in a field like journalism. Want to do a one or two day trial of a reporter? Pay them to write for you! Have them work in a real environment and do real work for publication.
If you're not willing to pay people for their time, and it's not an executive position, asking for days of a person's time is asinine. Also, if the homework isn't really real world, I still don't think it's of value. Making interview processes longer doesn't make them more rigorous. Only rigor makes a process more rigorous.
What I tend to do is have people do contract work for me, and if they do good work and people like working with them, I'll offer them a full-time position. Another way that companies get at this is with internships. They are a low-cost way to scout talent.
I find that a lot of companies are mistaking theater for rigor. If you ask someone to do a coding assignment, and it's not the kind of work that your company would actually do, how much value do you get out of it? Can someone work with your APIs? Are they comfortable with your languages and systems?
I have also found that longer interview processes that have more people involved introduce noise, not rigor. This is why I generally try to be involved in the entire interview process for people I am hiring.
Want to work on my team? I'll screen the resumes, I'll do the phone interviews and then I'll bring you in in person. I'll be in on all of the interviews you have with various people. It's my job to hire great people to work for me -- not recruiters, HR or random people at the company. I involve HR at the end to get a sense for a person's personality, not at the beginning.
As a manager, hiring is arguably my most important skill. A manager that can't hire is going to have trouble building a good team that works well together. So, getting the hiring process right is critical.
Most people are not contractors with an established company/legal status as contractors, so you want to hire them. To "hire" someone formally for a week, though, entails a lot of paperwork and has, depending on the country, legal consequences (e.g. voting rights in worker councils, minimum employment durations, right to sue for wrongful termination)...
The optimal way would be if lawmakers could create some sort of "employed contractor", however I can think of a dozen ways for employers to screw over poor people with it...
It's not like the job "Uber driver" where the fundamental nature of the job means you can't be a contractor.
At least in the US, this isn't a problem. You just make out the check to the individual, and then send them a 1099 at the end of the year. No need to have any kind of established company/status, because by default anyone running a business without incorporating or organizing is operating a sole proprietorship--it's basically 1 extra form at tax time.
As long as the person you're paying is doing business under their name, they generally won't even need to open a second bank account (they might want to if they do this more than occasionally).
Wow, that's awesome. What happens, for example, if someone is unemployed, then doing work for you and either gets sick on his own or gets injured on the job?
In Germany, for an employed person, the medical insurance would cover #1 and the Unfallversicherung (accident insurance) of the employer would cover #2. Let me guess: the (potential) employee is totally uninsured in the US during such an relationship?
For the first case--nothing at all.
For the second case--generally nothing as long as the person is really an independent contractor.
In the US there are guidelines that determine whether someone is an employee or a contractor. For example is the person flexible to work whenever they want, or do they have to be in the office from 9-5. Does the person have to wear a uniform etc. Is the position short term etc...
So if someone were injured on the job, they could claim that they even though the company said they were an independent contractor, they were actually an employee because of xyz.
Most employers know this so they take steps to ensure independent contractors actually fit the guidelines for independent contractor. And in the case of paying a tech contractor for a few weeks, it's usually pretty clear cut.
The IRS has guidelines on who is considered an employee. If you set their hours, provide equipment for them to use and there's a few other things I forget, they are a de-facto employee regardless of how you pay them: 1099 or W2. This is why a lot of larger companies will not employ sole-proprietors directly. They require you to either go through a contracting company or set up some corporate structure so that you can't claim to be one of their employees.
>This is why a lot of larger companies will not employ sole-proprietors directly.
There are plenty of large companies that do contract directly. For example, Coke does it all the time.
Is this problematic? Here in Portugal I just update my status in the tax authority website, then declare the extra income when filing my taxes (plus possibly pay social security for that month). You don't need to create a company or anything like that.
Most of the engineers that I've hired came from another job (not all though), and so they need a confirmed offer before they are willing to jump ship.
So this practice introduces its own selection bias; you'd only be sampling from the pool of currently un-/fun-employed, and those who hate their current job enough to quit without a safe landing.
I'd vastly prefer to have a week or two of paid contract work from each candidate before making the decision, but it feels insulting to even ask.
But I wouldn't just do this. I think any company of any size needs a strong internship program to build their own pipeline. I also feel that you should be able to hire people without them doing contract work for you or other kinds of assignments. Talking with someone, going over code and APIs, etc. is usually good enough to gauge someone. If someone has years of experience, has stuff on Git and can talk coherently about what they are doing and why, they probably can program.
What I was saying is that if you insist on seeing code first, pay them as a contractor, don't ask them to do a bogus assignment for free.
Unless you work on Google or something with the same size, more than you expect to hire.
And yes, you will biasing your candidate pool. Every kind of procedure will bias the candidate pool. Again, unless your company is Google-sized, the only important thing about that bias is if it align with the preponderant bias (what is bad) or not.
If you mean (which is bad) then I agree completely. You want your set of biases (biases being errors in hiring, meaning the things you select on that don't match with what the job needs) to be as different as possible from the rest of the industry. Whenever you can pick up people who are good who aren't getting picked up by the rest of the industry, everyone wins.
If you are not one of those companies, your focus should be on making sure you are not biased the same way as them. Other means of avoiding rejecting good candidates have a much lower return.
Maybe the lesson here is that the "optimal" strategy is to simply not abstract the problem. Its not "a person" you're hiring, its Bob. And Bob is more comfortable doing X or Y or Z, and Bob is best tested for his skills in ABC manner, etc.
There is _literally no chance_ that I am going to leave an existing job for a "week or two" contract with a company. If that's part of their hiring process, I will simply look elsewhere.
Maybe that's a Silicon Valley thing?
People might have clauses to prevent people from working on similar work for competitors, sure, but I haven't seen anything that bars working on other kinds of projects outside of work.
Though as you say your a journalist they tend to be exceptions.
But if you're working someplace salaried, it seems fairly dishonest to spin up a short contract. especially when most of the attention is consumed in the process of getting started (environment, implicit policies, platform, etc).
plus on your side, you don't really know how much of that person you're getting. if the work is great, thats fine, but if it seems slow or misguided, is that a fundamental reflection of ability, or because they tried to cram it in between 8pm and 11pm every night?
Companies dont do a week or two of contract work because people that have strong experience will just say no.
I find test paid projects are the best interview technique for detail oriented work, whether it's software or otherwise.
It's far better to select a task that is simple enough for the potential candidate to be a pinball through the code base or problem domain, ideally to solve a small problem that will be directly transferrable into the project they will work on.
I also make myself available to answer any basic questions or assumptions to save them time, because that would be the case on a normal team.
This results in a far lower expenditure of time trying out different resources to find relationships that work.
Unfortunately a lot of technical hiring is still done by non-technical people, or tech folks need to level up their hiring skills.
I wouldn't condone a practice like you've described. It's far better to engage a potential candidate at normal rates for a short period to see if it's a mutual fit.
But if you can work with someone before you hire them full-time, you'll greatly increase your hit rate. This may not always work, and it may not even work 50% of the time, but the more people you can hire that you have worked with either through internships or contract work or from previous jobs, the better.
Sure but you also filter out a ton of good candidates. I can't do my full time job plus family life stuff and work for someone else on a side project.
And I'm not going to resign from a job for a "maybe" contract to hire type position.
So it's just a non-starter. And most other programmers I've talked to are the same.
It's different for younger people who don't have families but that's exactly my point. You filter out whole demographics with the contract to hire style.
It seems like it's likely to work similarly to bug bounties: it's not like you stop getting security reports from people who can't accept the bounties, you just don't get the benefit of the incentive, and maybe the money gets donated to a charity of their choice instead.
These policies are actually fairly onerous as written. Most of us just get by because the company doesn't care to enforce.
So those incentives, I think, also apply to take-home projects. Those are also about as industry-standard these days as in-person coding interviews, and so any legal or social pressures against saying that you can't write code in person should also apply to saying that you can't write code for a few hours on an evening as part of an interview.
Meanwhile, the legal situation for "you can't get paid" is very different because so many legal things are different when money is involved, and the social pressures for "you can interview via whatever means they want you to interview, you just have to refuse payment" are going to also be very different.
FWIW, if I were one of the interviewers on OPs team, I will find this to be micromanaging and will be uncomfortable.
Doesn't journalism also have a lot of sites/publications/companies trying to get people to work for free full time? I know quite a few pass it off as working for 'exposure'...
The company in question advertises here on HN's "Who's hiring?" threads, but the role was brought to my attention by a recruiter.
Here's a little tip: If you expect to take more than 2 - 3 hours of someone's time, you should be paying them. Contract-to-hire and trial contracts are very common in the industry and most people have no problem with them. If you require a significant time commitment to interview someone then you should be respecting their time. I don't want to work for any company that doesn't respect my time.
I've never been given a homework assignment for a job interview, but I would love it. I absolutely hate the tests they give you on the spot, the last one I did was awful and made me angry! I would much prefer an 8 hour homework assignment. That pressure would be fun, and I get to work with my tools in my comfort zone.
I spend 8 hours playing video games (over a few days), so putting that aside for a challenge with possible good job at the end seems okay to me.
I think most people are ok spreading 8 hours of work over a couple weekends. Paid shows the company is more respectful of your time, but in the end if they're taking up much of your time they should absolutely be flexible, or they will definitely pick-up a selection bias in their candidates.
To me, this means the company does not value my own time, and it might even be a red flag which indicates unhealthy expectations about your work-life balance.
If they had an assignment which would take about an hour, maybe two, fine. But any more than that and I think you should be paid for that.
To be clear: I've been given take-home "challenges" that take 2 - 3 hours each. Including at the company I wound up going with. I have no problem with these. If you expect me to sacrifice a full day of my time then you can pay me or find someone more desperate than I am.
The problem becomes, but then when would play videogames?
It's fine to be eager for an occasional puzzle challenge, but don't forget about the opportunity cost. They want 8 hours of your videogame time for something that likely has a low return on that time investment (ie, the homework assignment is [not] productive code; there's no guarantee you get the job as that homework assignment is only input in the process, etc).
That sort of opportunity cost favors the young (with no or small families, and a stronger ability to live on fewer hours of sleep), the eager (passionate for that particular job), and generally those with fewer work/life balance concerns (what if you need those 8 hours to stay healthy or protect your family?).
If you are already employed, you risk burning out by trying to burn the candle at both ends and do unpaid work in your off-hours. There's a possible opportunity cost that your day job's productivity suffers and you potentially lose your job.
Even when unemployed there are opportunity costs at play (if two companies need 8 hours in the next 12, which one do you turn down). The old adage is that unemployment is a full-time job (hunting for prospects, networking, talking to recruiters, dealing with unemployment wage systems, etc), and is stressful enough without adding additional unpaid labor on top of it.
It’s not “instead of” it’s “in addition to”
1. Initial phone screen with recuriter - 30 mins
2. Phone screen(s) - 1-4 hr
3. (optional) phone live-coding on coderpad - 1 hr
4. Take home assignment - 4 - 8 hrs
5. Full Day onsite interview - 2-3 days if traveling.
Rinse and Repeat for atleast ten interviews minimum you are looking at almost 1 month of pure interview time. Add on practise time for algorithms and puzzles.
Absolutely not. If most applicants need to do 4-8 hours worth of homework, they'll never have the time to review the results for every applicant. And making me do 4-8 hours of work without getting it reviewed thoroughly and getting personal feedback is not valuing my time.
In the process above there already was a screen sharing coding test. Adding the homework on top is redundant. You should be able to tell if someone can code in one hour of screen sharing, but you can't really tell if someone is mediocre or top notch from a single homework assignment.
Only once have I been given a homework from an interview and it was a solid 10-20 hours worth of work. I refused to do the work and I got the offer anyway, but I turned it down in favor of another offer.
It's much lighter to schedule than a phone interview that requires a 1 hour time slot with everyone during working hours.
Ignore homework that takes 10 hours to do. It's abusive and it only selects desperate people with no family.
Do you want to work with team mates who have been vetted over less than 2 hours? I've worked with people who couldn't code, it's not a fun experience.
There are a lot of people who can talk a good talk but can't actually code. This was the reason for the FizzBuzz test (which I dislike, but it was invented to solve a real problem).
My current employer would not be able to do contract-to-hire or a trial contract for legal and policy reasons, and I strongly believe that a homework assignment taking approximately 2 hours, when combined with another 1.5 hours in person provides the best signal of all the methods I've tried. That's a total interview process of 3.5 hours i.e. about half a day. If you can find half a day for an in person interview then you can certainly find the same hours split up differently for homework + in person.
Trying to make it about sexism is crazy. Homework is much more flexible as to when you do it than in person interviews. Just because you're looking after kids doesn't mean you won't be able to take a couple of hours when they're asleep or in school. If doing homework is a problem, then I'm sure doing an inperson is even harder.
Our homework + inperson extension is on a topic related to our real field, not just a compsci quiz (which means it gives you a chance to show off domain knowledge if you want the extra points), allows no frameworks (except testing), produces an interesting artefact at the end, and can be completed in less than 100 lines of functional code plus a smattering of test code (although many will do more, sometimes crazily so - for which they lose marks). There are exhortations in the instructions to keep it simple. You can schedule when you want to work on it yourself, work on it with your own tools and hardware, split the time over a couple of weeks if it's difficult to get a block of time together.
The only real question is the time commitment. I've targeted 2 hours, but many spend around 3 or so, and some spend more on it. Setting an 8 hour challenge for my kind of work woud be entirely unfair to the candidate, unless that was the total time spent on the entire interview process.
I've hired with a number of different processes, and this one takes the least total time for the candidate and seems to have given me the best results.
The problem seems to be with unpaid 8-hour challenge.
Recruiting is not a cost-free process to begin with, and perhaps this is not the right place to cut these costs.
The article makes two good points re: bias and lack of research. To the first I think it's up to us as interviewees to mention this in all of our calls. "Do you have a non-take-home option? For some groups this could be an exceptional burden."
To the second there's no real answer other than a lot of companies need to try it out to see how they fare and then when they have a basic idea of logistics that work run a study on efficacy / predictive power.
In this case, I don't see it as unreasonable, because you didn't have to do the project just to have a shot at someone talking to you in person.
I would never do a homework project as the first step in an application process. I would do one as the last step, after I had learned enough about the job and the company to decide that I really wanted it, and they had narrowed down the field to me and a handful of other candidates.
When I was job hunting last year, the steps typically went like this.
1) 30 min phone call
2) Homework
3) Algorithm Exam
4) Onsite
I typically got rejected at #3 which means the homework meant nothing.
It got you to step #3, but if I were applying to that company, I would not have done the homework. When they told me of the homework assignment, I would have asked, "assuming I do it, what comes next?" And when they replied "an algorithm exam, followed by an onsite interview", my reply would have been, "homework is the last step, or no thank you."
Then again, I have almost two decades of experience and a well-established track record of accomplishments. And I currently have a solid, well-paying job. If I were fresh out of school, trying to land my first programming job, I would probably be more flexible.
I don't consider myself a rockstar, or even a prima donna type who would make "demands" of the interview process. But I'm confident I could find a job fairly quickly without jumping through ridiculous hoops just to get an initial response from a company. From my perspective, the software developer market is still a seller's market.
YMMV, of course. Maybe you're in an area where there are more qualified developers than available jobs, and employers have the upper hand to the extent they can pull off bullshit like this.
The hiring market is strongly an employer’s market. There is far more talent out there seeking relatively few jobs, so employers can dictate terms pretty much absolutely. If they require a 5 day homework, then it’s “homework or GTFO”. If they require whiteboard hazing, it’s “whiteboard or GTFO”. The last real employee’s-market we had was in the late 90’s!
Wow, are you in the U.S.? This is pretty much the exact opposite of my experience and perception. I'm sure it must vary somewhat based on your specialty and what kind of work you do.
There are entire deserts where there isn't a tech company in a hundred miles around.
Even in the most active tech hub, it may be difficult to find a new job, assuming you care about things like work hours, health coverage or a moderate commute.
I couldn't agree more on companies being picky. My wife interviewed for a company last year, they said that they interviewed more than a hundred candidate so far in the year and they didn't hire anyone.
It's fair to say that they are not recruiting for real.
I really can't call everyone who has a decent resume, its way too time consuming. If we had a recruiter then maybe it would be worth it, but not personally. Maybe a take-home quiz that takes like 20-30 minutes would be a good filter instead?
Even if you have, say, 100 resumes you consider "decent" (which to me would be an impossibly high number) then you need to be able to rank them, so you can pick, say, the top 10, and invest some followup time in those.
Amen. There is no shortage of applicants. But there is a shortage of competent, qualified ones. In fact, I find it frightening the degree to which blatantly incompetent people can skate by in our industry. I recently interviewed someone who rated themselves as a "professional level" Java programmer. (The highest level he could have rated himself is Expert, one above professional level.) The first technical question I asked was, "OK, please write a Hello World program in Java for me.". He totally bombed. It was painful to witness. He had no clue what he was doing, but yet, there he was, asking for a six-figure salary and deeming himself worthy of it.
The extremes:
Is there a shortage of talented engineers willing to work for minimum wage? Of course!
Is there a shortage of talented engineers at a price of, say, $5M a year? I guarantee not.
So there exists a market wage somewhere in between at which supply of talented engineers must equal demand. If you think there is a shortage of real talent you might not be at market rate for that talent.
Using your example, there exists a market rate for an engineer who doesn’t know how to program Hello World. You might be surprised how much that is!
Wouldn't it be funny if the half that didn't respond were the best candidates you heard from, and you threw them away with your take home thing?
It's the irony that the thing you did to make sure you got good applicants was the very same thing that turned the best away.
At some point you need to ask the applicant to do something to prove their worth and interest in the company. Take home tests are only one filter for that and I'm honestly open to others.
In my estimation, it's not likely that all of the ones who didn't respond were better than the ones who did, but it's likely that a sizeable portion of them were. They're not going to jump through ridiculous hoops because they don't have to.
One time an employer of mine wanted newly hired developers to sign a crazy onerous contract with unbelievably broad and vague non-compete provisions and IP provisions. I told him that the only people who would sign it were people who were desperate for employment, or people without integrity who would sign it without any intent of abiding by it. Neither of which were the types we were looking for. So he relented and we ditched it.
The exercise is sent upfront. A good candidate can estimate that it's a reasonable program doable in an hour. It's up to him to decide whether he wanna continue the application or not. Either way, both parties win.
A bad candidate who can't code cannot return anything. Little risk of false positive.
I think the only important thing is to keep the exercise short enough, one or two hours top. It doesn't exclude people who have a family or other obligations.
Then again, I never took any "algo class" or any other CS classes, so what do I know. I just think people fret way too much about getting all the details right in coding interviews, because in my experience (both interviewing and being interviewed) that just isn't the point.
I've seen varying schools of thought where a 3 day time limit implies you're expected to spend full workdays (3*8 = 24 hours) on it, which is less at your own pace.
I'm still annoyed at a 2-day assignment I was given which could not be done in less than 16 man hours.
My life is separated. I have a computer filled with dev tools at work but my home computer is a media center plugged on the TV. I do have a laptop for emergency work but that computer is provided by my employer.
I wouldn't do the job interview on that computer.
Whenever some company tells me to do homework as part of their hiring process I simply run away. What this tells me is that they expect their employees to do unpaid work at home.
I wasted a week coding up a ML paper but never heard back from the interviewer. At least in white board interview they spend time with you with real time feedback.
But the whiteboard test should only be a smoke test for can/can-not program (ie. test for impostors and frauds), not an assessment of aptitude or in-depth knowledge about algorithms. 10-15 lines of code or so, "Guess the number", "FizzBuzz", "Count zero bits in 32 bit int", etc.
I pushed back at the contract, and asked them to ask their lawyer if that was really necessary. When they insisted on a contract for the homework project I pulled out of the process. It just felt so wrong.
The worst part, is not that they would seek some way to freelance a small part of their system, nor that they would actually use ideas from hiring interviews in their product -- but that (it seems) they would so blatantly lie to someone they professed to want to hire that this was homework, when in fact it was not. Even worse was the feeling of disrespect that they wanted to freelance part of their system for USD450, trying to abuse the hiring process/ talking a big game about offers, to unfairly lowball the price. Like they are a well funded, established, largish YC alum, at least they can afford to pay market. Anyway, everytime I see "XXXX is hiring a YYYY in ZZZZ" on here from them, I'm reminded. Feels good to vent.
Just sad. Anyway, I'm not going to name names because this is unwise to do that in a public forum.
Anyway, I only added the above anecdote after writing the following comment.
This is why Triplebyte is so good. You do a single technical interview ( after a quick online quiz ), then get fast tracked to final interviews at various YC / silicon valley startups.
Sure, maybe not everyone wants to work for a smallish startup. But some of their clients are 200+ people. So...it's not all the same.
When Coinbase first launched, someone was having issues getting funds out of there. That person made a thread on HN, and the response has always been “HN is not a support forum for YC companies” by the moderation team.
I see a few people seem interested in the idea of naming such companies. I have wasted a few weekends worth of my time (which at freelance rates is a decent amount of money) and have been ignored when I asked for feedback (sometimes just ignored completely). Its disrespectful and companies should be called out for it. I could name at least one one company that regularly posts in the hiring threads here.
The HN post is here: https://news.ycombinator.com/item?id=16895043
Pretty simple (beanstalk / elasticsearch, no sessions): https://news.ycombinator.com/item?id=16895043
Hope you find this database useful.
I would point out that on the hiring side, evaluating a submitted project in a way that can differentiate between good and great candidates, and that the reviewers can understand well enough to get the signal they need, is very difficult.
Because of this, I know several times we have asked candidates to build a simplified version of something that we have already built internally, where we understand the problem space deeply. We only ever do something we have already built, as that's the whole point, and we typically simplify it, although that might not be obvious from the outside. We also cap the time at 1-3 hours depending on the task as we care about evaluating the candidate, not getting a complete/usable/production ready solution, and so that it's not dependent on how much time a particular candidate can spend on the problem, which would introduce biases.
This isn't perfect, and we try not to do this, and to use problems that are obviously toy problems, but sometimes it's necessary.
Only if the contract is signed.
Just a minor bit of friendly feedback on your comment, I hope you don't mind...
You said "Any new insight he might bring". To refer to a hypothetical developer as "he" contributes to the stereotype of developers being male, and that can make those who don't conform to that feel less welcome in the community.
I'm sure this wasn't a conscious decision, and I know it's something I slip up with frequently, but I'm trying not to do it, and I think writing that is more inclusive is more persuasive to more people and generally better.
Why not? create a throwaway if its a matter of coming back to you. Otherwise we really need to name and shame these companies if we have any hope of getting the practices to improve. Sunlight is a great disinfectant!
Ok, I created it: https://news.ycombinator.com/item?id=16895043
I'm replying you because you were interested in this ( or seemed to be ).
I thing is I don't think anybody even looked at what I made because to view everything in the app required the user to login (part of their brief) but no new users showed up on the site at any point. Pretty frustrating experience overall.
That being said, I would add some constraints:
1. The 'task' shouldn't take more than 30 min to understand, and no more than 1-2 hours to do. If it takes more than that, you should make some obfuscated dummy-code endpoints that do everything you're not directly evaluating on.
2. You should let the interviewee know that it should only take 2 hours as well. If it takes more, then there was either a misunderstanding in the task, or the interviewee doesn't have the right knowledge/experience/qualifications etc.
3. You should do your homework as well, meaning you've thoroughly reviewed the code submitted and have detailed questions ready to go for the interviewee. This should also be the 'control' to make sure the interviewee really did write the code (you'll know really quickly if they didn't)
4. No other coding tasks in the interview process. You can have them review code or suggest approaches to solve problems (things that people do on the job with others)
5. The interview should be shorter than if they were asked to do whiteboarding or live coding or anything like that.
Most employers have (on paper) "trial periods" of around 3 months. A 1-2 hour coding assignment followed by 1 hour of well-crafted questions about the assignment (plus the usual interview stuff) should be enough to know whether or not you want to give the employee a chance for 3 months.
(I realize most employers don't use the evaluation period for evaluative tasks, they just have the person start and hope for the best, but if done right, it's actually a pretty good system)
Just last week, I was interviewing for an opportunity as a VP of Engineering for a post-early-stage startup. The CTO gave me a take home assignment involving building not one but three complete applications (to be delivered in a docker container so it could be easily run). When I told him that I didn't have the kind of time it would take to perform this, he told me all the other candidates for the same role had no issues with it.
I ended up withdrawing from the process.
If senior roles itself are being hired like this, then I can only imagine how much worse it would be for individual contributor engineering roles.
So yes, I suppose in a way it was not a culture fit for me, and that was indeed why I ended up withdrawing. I would also urge anybody interviewing for any role to evaluate whether a company which requires you to spend a substantial portion of your personal time, before you're even an employee, has the kind of culture you're okay with.
This is great for companies who are willing to sacrifice getting the best candidate for assurance they won't get a bad one. This makes sense for a small startup, where a poor candidate hurts growth more than a great candidate helps it, but less so for a Google-class company where a single great candidate is like a billion dollar lottery ticket and the bad candidates cost almost nothing.
1. A lot of studies out there show that the most important factor for a hiring success is culture, not skill.
2. Are you even measuring skill with your test? It doesn't matter if it should only take 2h for a candidate to complete your test, if they're nervous or want to impress they're going to spend 12h on it. How can you tell the difference?
3. What about the really good candidate that doesn't want to spend 2h on your test, because he also went to 2 other interviews in the same day where no tests were demanded of him. You're again not testing for skill, but willingness (and against other employer's testing regimes).
4. What is your test about? As a random example implement Web Service X that does Y. Is it important for you that candidates know how to implement X from the ground up? Will they be doing a lot of Y? Why spend 2 hours effectively answering 2 questions, where you could have interviewed him in 2 hours covering his whole history of computing experience, what he loves about it, why, what code he's written, what he's into, what he thinks about X,Y,Z etc.
5. Your job is not to impress him with your knowledge (not that I'm saying this is what you're doing, just a general remark on interviewers out there), but to determine what his is, where he's strong and where not, and how the person will fit into your structure. Making your test the big determining factor is a failure.
6. There's a reason fizzbuzz for example makes a great starter test. It immediately weeds out people that can't complete it, and allows you to see the interviewee reason through it. He goes for example, well, I take this and that, then I have to do this; oh yeah and so so etc. It allows you to see a lot more than just the end answer. After that you can apply more interesting/harder questions, but focus rather on how they do it than if they can complete it in the allotted 5 minutes.
What we administer is like a web app equivalent to FizzBuzz in terms of difficulty level. But unlike a whiteboard FizzBuzz, it doesn't put the developer on the spot. As an extra bonus, unlike FizzBuzz, you'll get different results from the kid who just successfully finished a CS degree and the grizzled veteran who's built and maintained complex systems for decades.
The right challenge can be completed in an hour or so by a mid-level developer in a brute-force manner. It will weed out those who simply can't code (or don't grok how the web works or how apps are constructed); and also highlight candidates who are capable of more sophisticated work. If being up to speed on a specific technology from Day 1 isn't important, you can give the developer their choice of framework or language or whatever.
Then when you do a code review in person you can get a feel for both how the developer approaches code (a skill) -- but more importantly, how they take critique. Do they get defensive? Do they riff off your ideas? Do they suggest improvements/refactorings they would have made if they had more time?
You can also explore other cultural factors - for instance, how did they approach constraint trade-offs in terms of time vs. completeness vs. sophistication? How clearly are they able to communicate about technical concepts?
It also got me thinking: what if you just had some generic code, which was low quality, or a prototype front end app, and then asked them to refactor or code review it? That could spark some interesting discussion and be less stressful, but may be easily bsed I think.
No! Studies show that above an sufficient baseline level, having a baseline of soft skills matters more than marginal differences in tech skills.
I start with a phone screen. Then onto a (relatively short 45 min to an hour and a half) face-to-face. This is a 'let's get to know each other and our expectations' low-stress interview just to make sure we understand each other and what it is we're actually getting into.
Then, a technical discussion. Approaches. Experience. Past projects. Some 'did you lie on your resume' questions but nothing that would require a whiteboard or anything. This culminates in getting a paragraph of requirements that will lead to the take-home test. The requirements are intentionally vague and missing critical information to see if the candidate recognizes this and asks the right questions (I'm convinced that 60% of any job is asking the right questions). Then, when I feel like they've hit a few of the important gaps I pull out the detailed specs that tries to answer all of the questions they might have, serve as a reference sheet, and make it clear what not to do (like turning a two hour task into an over-engineered weekend project just to try to impress -- I've got to review all of this code after all!).
This is not a real world test but it is a somewhat simplified version of a real-world thing they will be expected to do. It shouldn't take more than 2 hours.
I point out that if they have any code they can show me from a real, working project it can act as a substitute. What I am looking for is knowledge, approach to the problem, and code style.
Afterward, I am looking for how they handle constructive criticism.
This has saved me and I think applicants so much time with much higher quality information than a long interview or a nerve-wracking whiteboard session.
I give them five days to a week to finish. Most do it within a day or two. If a busy person can't find the time that they'd normally spend in an interview session essentially any time they can fit it in over the next week or so...well, too bad.
Oh, I also give them explicit permission to post their solution on github so that if we don't end up hiring at least they'll have some code to show the next potential hiring manager.
The candidate thinks, what's the point? It's a seller's market. I'll work somewhere that doesn't haze candidates.
We run work-sample challenges now, and (loudly) did so for about a decade at Matasano. We had essentially no complaints once we locked our process in. Part of the reason why, I think, is that we tried hard to be very clear about our process. But more importantly, the work-sample challenges made most of the decision. When we told candidates the time they spent on the challenge offset time they'd have to spend in person performing for an interviewer, we weren't lying. A 4-5 out of 5 average on our challenges more or less totally technically qualified a candidate; the odds were overwhelming that such a candidate would get an offer from us, and we told them so before they came in for interviews.
What I think I see now is hiring teams who want to have things both ways. They don't want to take a leap of faith with a new process, so they're still abdicating their responsibility to random subjective interviewers. But they want to look on-trend and competent, so they assign homework.
When you get on the phone with one of these teams and want to check them out, ask about the rubric for the challenge, and ask if they're going to do any on-site technical qualification after you do them (if they're giving challenges after the interviews instead of before, hang up the phone on them.) I'd be surprised if even half the companies that assign homework even have a rubric.
I spent a few days on one, sent it in and was told that day that they had made an offer to someone who accepted the offer. Then I did another one over the course of a few days which was looked at, but a new VP came in and eliminated the opening.
It's similar to asking me to come down for an interview for a few hours during the work day (and then to come back, because one person I had to talk to wasn't there). Or before I've even talked to anyone, they hand me a form asking for three references, preferably (or exclusively) former managers, and their current phone numbers.
I don't mind them asking or me giving these things if they are near a decision about me, but almost every short phone screen I have nowadays is followed by a request for a work sample that in reality would take a couple of days to do (between my schedule, and wanting to send it in near-perfect).
Something I think is preferable is for them to ask me for code I wrote. Then I can write something once and be done with it.
As others on the thread have said, in a tight market, companies put up huge new barriers to getting a job with them like several days worth of free work, then bemoan the fact they can't hire qualified (young, SF local) people who "care about their mission" of selling advertising junk from one corporation to another corporation.
Nowadays, I reject more companies than I interview for based on an initial phone conversation. I have requirements in what I look for in a company and position, and some simple questions help me resolve many of the primary requirements. Given a take home exam, I will have invested hours of my time, before given a chance to screen the company.
Just say no. End the madness.
EDIT: I should add that when I mention a phone conversation, I meant with someone on the dev team, not HR. A take home exam after only an HR phone screen is not acceptable.
there are lot little details that one can ask and will give you back a picture of how the working environment is going to be. here's a good starter list, even if I find it too formulaic and question you need to ask heavily depends on what your role will be: https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s...
also, as an interviewer, if a candidate doesn't ask about the work environment and daily operations I'll take it as a potential sign of the candidate having never reflected on its own craft.
When we gave homework, it was to a slate of junior candidates, primarily just out of college. When I did homework, I was unemployed and looking for my first job in tech.
If homework is a good idea ever, those are the circumstances. You have candidates who are hard to evaluate, will get a large benefit from a job offer, and are less likely to have severe time constraints. I wouldn't make it a requirement for all potential hires.
As you said, we calibrated our general expectation to what we think is reasonable in 2 hours - so we expect "good enough" code, but not perfect code.
One of the biggest mistakes I see strong technical candidates make in interviews is not asking any questions at all.
The code itself is, of course, valuable for helping us to evaluate, but honestly some of the best signals we get come from the open-ended discussion that results from us talking with the candidate about their code when they are done: "imagine this file you're processing is 100TB instead of 100kb? Would you change anything?" or "What if this clean data you've been given to work with had this or that issue, what might you do?"
My only point of disagreement with this article is that it seems to assume that a homework problem has to be really time-consuming or burdensome, but there's no reason that has to be the case. To me employers who are asking candidates to spend hours or days on homework are of a piece with ones who ask people to solve tricky, unrepresentative computer science problems on a whiteboard: they reveal that they don't really know how to interview people and they don't have much respect for their candidates.
Even with rehearsal I am not a great public speaker and impromptu formats certainly don't make me better. I also prefer take home assignments since it is more closely related to how I actually do work: Get a problem, think about the requirements, do some research and design a solution, implement.
Since I personally prefer this style of interviews, I have given assignments like this. Usually a simple microservice that is given AWS t2.micro size constraints and no time limit for a response. When I get a submission I am looking at code style, tests, and documentation as well as solution quality to get a better picture of how a candidate approaches tasks and gauge how they work independently. It's amazing how much you can tell about a single person from a challenge like this.
I also think it is more flexible since there are no strict pass/fail criteria. A solution in C/C++ that optimizes heavily but skimps on documentation could be just as impressive as someone who uses Python to stitch together multiple tools with more features and more thorough documentation.
Interviewing is difficult...for both interviewee and interviewer. Finding the right compromise between depth for the interviewer and work required of the interviewee is a challenge. I am sure there are take home challenges that are ridiculous, but I wouldn't dismiss all take home assignments.
I'm a general contractor (Unicorn Homes Inc.) and I'm currently building a number of spec homes. We're going to need electrical work done in all the homes and I'm in the process of sub-contracting out that work. What I need from you, Joe Electrician, is for you to go wire the kitchen of one of these homes. Once you're done I will inspect your work and let you know if I'm interested in hiring you for more work.
So it's more like "What I need from you, Joe Electrician, is for you to go wire up this demo kitchen frame that I've set up. Once you're done I will inspect your work and let you know if I'm interested in hiring you to wire up the actual houses I'm building.".
- A short phone interview using a collaborative editing tool like CoderPad.
- An example project they can complete independently and turn in. We've designed this project to take about 2 hours to complete.
Almost universally, candidates choose the "take home" project option.
Then you've got it right.
It's good to see people admit whiteboard interviews are a crummy system, I think you can learn way more from X hours of "do a task on a real computer with the internet" than you can from the same X hours of "do algorithms work on a whiteboard".
The objection, for me at least, is to "do 10*X hours of work on a real computer". It's bizarrely common to massively scale up the time investment when moving to this sort of task. And the answer is so easy - just don't do that!
It's usually not "build out our new feature", because spinup time is a thing, but some companies do seem to throw around "build our new one-off microservice". Stuff like "given this incoming data, set up intake, a storage layer, and a backend layer that applies formula X". If the data and the formula look business-relevant, it's at least enough to raise an eyebrow.
That said, I'm not sure how much I care, provided the interview is sincere and not some kind of bizarre scam to farm out work. My main concern is much more with scale. I totally see the merit of replacing 5 hours of whiteboard interviews with 5 hours of "build a toy project at home". I don't so much appreciate "take a week of your life and build a substantial project so we don't have to do any legwork".
What I've seen is something like "this is a startup in market X, whose product doesn't do Y but could, and they're asking candidates to write code that does Y, and sign over all rights to everything provided during the interview process".
Optimistically, it means that if whoever they hire will be told "hey, first assignment is Y, you've already made progress!" Pessimistically, it goes in the same category as "Whartonite seeks codemonkey" and "sign this NDA to interview" - it's a matter of ego and ambition exceeding skill.
If software were like homebuilding, there would be legally enforced licensing.
Would it be so bad?
As an employer, hiring someone new is always a leap of faith. Might work out, might not. If it doesn't, dont feel bad to fire people again.
I can only confirm this. I used to be a freelancer and I got almost daily offers per email and phone. In order to stop I ask them a very hi price per hour ( 90-100 euros ) and to my very surprise yesterday the guy said, "that's fine, when do we start?". I backed up a bit and after some discussion he told me that they really could not find anyone. There is really no IT people in Germany without job, so the only way to get work do is to turn to freelancers and offer a good rate.
Contrary to what you seem to expect, that price is a steal. A more realistic rate for contractors doing basic IT work (like configuring DB backups) is 400-800 euros per hour.
Mainly it’s just a talk to see what personality is the best fit. Sometimes we do a personality test, but only for people who will do any form of project management. We’ve never done a technical test or had anyone program for us, I know some companies do it for juniors that are fresh out of school, but otherwise it’s faily uncommon.
That’s it really.
I’ve hired around 20 developers like this over the years, none have failed me. One junior wasn’t really worth his salt at first, but he learned quickly.
When I read about these American hiring practices I wonder why anyone would want to work there. I wonder if it’s because a lot of American developers are autodidact? I mean, I’ve never interviewed someone who wasn’t educated in some form of CS, academy level and up.
They made me do coding problems on the spot right there. Given an integer n, write a function to print ever digit backwards. It took me a minute to think about it, but i solved it, and evan ran it at home to make sure.
Several months later they ask me to do an online 3 hour interview. I did it, there was no human interaction whatsoever. I had to record video responses and solve code problems.
More than a month later with no follow up, i got an automated response to take another 3 hour interview with no human interaction.
I did not take the interview and will never re apply to IBM
They originally sent me the work over Skype. I don't generally log into it Skype but I did in order to accomodate the way they wanted to chat (send the work). A few weeks after they told me they'd hired somebody else I logged into Skype again to find a message asking how to access the work I'd done from several weeks earlier. Even though I'd provided a link to it earlier on in the conversation. To this day I don't know if anyone even looked at the work that I spent a full work day doing.
Perhaps they just didn't like the work that I did. I wouldn't blame them, I wasn't particularly proud of it. But I had a lot to say about the work, the problems I had and the things I would do differently if I were to start it again. I wasn't given the courtesy of expressing those thoughts though.
The funny thing is, they approached me about the position in the first place.
I wont be participating in any "interviews" structured anything like that ever again.
That said there needs to be a balance where the tasks are enough of a test to give indication of quality of work without taking 3 full time days to finish.
A sixty minute screen share is the sweet spot for me. They get to use whatever environment they are familiar with. Seeing someone work in their environment also tells you potentially a lot about the candidate.
Staff engineer that can't navigate their debugger?
Senior engineer that codes by copying every line off of stack overflow and mashing the keyboard until the lines somehow work?
Bootcamp graduate that builds test cases in Excel?
Junior engineer that writes code in notepad.txt and pastes it into a Python shell?
All things that I've seen happen that tell a different story than a whiteboard or a 3 hour submitted homework test.
(The candidate was previously in the accounting biz, so his navigation of Excel to do this was wizardlike to produce a few dozen test cases in about a minute.)
Work is going to involve some bullshit. That's life. If you can't jump through a few hoops for a job then you don't want the job very badly or have an ego problem.
Don't get me wrong, expecting someone to spend three days on a technical solution is nonsense. But the "special snowflake rockstar" mentality of some developers is not helpful and important to check for during an interview process.
One problem is that assignments take too long and have over-optimistic estimates, which apart from taking up time can lead to feelings of self-doubt. Feedback on the assignments is lacking to no-existent, and this is an unforgivable sin in my book. Also, anything that looks more like a contribution to their own software than an academic test is a red flag.
The other options noted in the article are also pretty bad. Whiteboards and fibonacci numbers, forget it. "Pair programming" in an interview is deceptive, nothing like normal pairing; it's a test (fair enough, but don't call it pairing). It's also problematic for me because I have a problem with my eyesight and any "accommodation" requested up front tends to be inadequate, leaving me at a disadvantage.
For people with busy lives or family commitments, a short "interview prep homework" of 30-90 minutes would help the person to prepare, in their own time, in a relatively less pressured environment. The interviewer could then expand on this during the interview.
I've done assignments like this where it proved a useful opportunity to brush up on a particular language (if cross-training or coming back from management), sometimes even enjoyed it. Also, it means recruiting is less of a numbers game - you might interview with a handful of companies instead of a dozen, choosing the ones you really care about.
IMHO what we're seeing is a terrible implementation of a reasonable idea.
Went in there, did their pointless psychometric tests (which by the way litterally have no relevance to my skills) went through their interview and went out their door passing it with flying colours.
I got a call from the agent leasing with them. She was shocked to say that neither me or another they interviewed made the cut.
I graduated with a first class degree with honours in Software Engineering.
The company said I wasn't qualified enough for the graduate position!
Told a good friend and lecturer at my university about this and he said, "pass me the name of the company, i'll get them barred from exposing internships and job oportunities to other students. Your time was utterly wasted and that was a completely pathetic reason to not accept you".
The agent said to me that she would fight to testify my qualifications. I told her that there was no point. This company is not only wasting my time but your time as well. They dont deserve your time as an agent nor my skill set.
Social skills and having a compatible personality with others on the team is very important to team success too.
May have been something else but you saying "did their pointless psychometric tests (which by the way litterally have no relevance to my skills)" makes me suspect you failed for non-technical reasons.
So many companies have these assignments. Theyre infinitely better than whiteboarding, or hacker rank tests, or "pair programming" on some crazy abstraction for only 30 minutes. Some of them are so much work - I am working on my 20th hour of a really involved full stack project, and still think I have about 6-8 more hours before its "pretty".
Then I have 2 more to do, and I told the HR people at those companies Id do them 3 days ago.
One weekend I spent 12 hours doing a SOA example project, and on Monday, the owner simply deleted the upstream - and my fork along with it - and said nothing back to me. I hadnt even turned it in yet!
Hours, weeks... its been months of this. Im working for free and its just energy that goes into thin air.
Not once has anyone actually done a real code review, I suppose they just get buried in hundreds of these little projects to review and select someone and just never get back to you :-/
Next week I have about 6 interviews lined up. Cant wait to see how many unpaid hours of work I get to do!
...oh, and I have > 12 years of industry experience, have built multiple applications that garner 50mm requests per month (I can almost guarantee you, that youve been on at least 2 of my websites before), have worked with IoT, HIPPA medical, ecommerce, greenfielded, been on big teams, worked internationally, have buttloads of experience, and every person Ive ever worked for praised and lauded me plenty. Yet I cant get a fucking job for the life of me because of these paint-by-number processes.
Yah anyway, back to the grind, for free...
Consider that for most jobs, there is naturally a 98-99% rejection rate. Only one position and 300 candidates. So assume that most coding exercises you do will go nowhere, will not be reviewed, and will leave you feeling disappointed.
You are a thought-worker, and an obviously talented one. The most important thing you can do is to stay nimble and don't get discouraged or bitter. Part of that means saying "no" to unreasonable requests. You don't have to full out say no - you can also just negotiate down the scope to something digestible.
Identify your limit and push back should anyone attempt to cross it. For me, if it's a good company and the test is reasonable, i'll throw in 2 hours. If it's a BS test, I'll push back or skip entirely.
These companies hiring with absurd processes do not know what they're doing - and will walk all over you/us if given the opportunity.
The other one was for Automattic that seemed to be intentionally vague to check if I would spot some pitfalls and smells they left behind. Never heard back. Mailed the guy that I had contact with twice. Not a fucking word. Thank you for wasting my time.
I don't think I'll ever do it for free again unless it's a position for Godmaster of Cooltech at AwesomeCo.
They asked me to write a GAE based web app for something or the other. I told them (politely!) that I'd do it, if they provided me with their diversity policy, specifically with an explanation of how this sort of exercise didn't discriminate against anyone (as noted in the article.) I got a "sure, we can do that" and then never heard from them again. I didn't consider that a loss.
Yeah you dodged a bullet there. Digital design agencies are almost always sweatshops.
Not only that they gave me a vague reason, only after pressing for feedback.
Left a very bad taste in my mouth, don't think I'll be doing code assignments again, what a waste of time.
Then give up to 3 months of mentorship. If the person doesn’t look like they meet expectations, then fire her quickly. Give them the remaining time up to 3 months as severance.
This is the best way to do it because you're not subjecting people to bad interviews, and you're giving them up to 3 months severance if it doesn't work out.
> Holiday’s company [CallRail, Atlanta, GA] now uses a structured “live-coding interview” format that involves reviewing code together and talking about it. The process typically takes between 30 minutes to an hour.
I wish there were more details, like whose hands are on the keyboard, or whether the interviewer has a rubric to structure their evaluation.
The only way I've found to reliably hire is a coding test. I think it's ethically irresponsible to choose actual client work for the task. But it is important to create a test that is similar and with a time constraint, especially if you work on any projects with deadlines so you can simulate how they work under a time crunch.
My tests took about 10min to write and 1-2hrs to complete and then we could do an apples to apples comparison across candidates.
My concern about client work would be the time it would take to bring someone up to speed as well.
It would be legal (though the ethics are still questionable) to give someone work after the client has decided not to give you a contract, as then there is no value in doing that work.
This is why most big companies prefer to to hire contractors. You get someone for 6 months, if you like them you hire them, if you don't you let the contract expire.
I'd much rather do a timed coding test for 30 to 60 minutes and be done with it.
At my current company we get them to write code (or something like pseudo code) down on paper. We give them a print out that clearly explains the task, which requires no domain knowledge and then leave them to it for an amount of time to write stuff down on paper. We then go back in the room and get them to talk us through their solution. But so many developers just can't do this.
Then some can, and they can't explain the edge cases, what if the array is empty? What if it's null? What if it's all the same value? They'll just freeze. They would have past the phone screen first to get this far.
We're not looking for more advanced stuff with red–black trees or an exotic data structure. Just a demonstration of what you've written and simple understanding.
In reality, this is an invitation to a conversation. Why did you do it that way? Is there another way to do it? Which way do you prefer? Why? All if this is basic first principle stuff.
Generally when you get to the good candidates you know. They can answer all these questions and you go in different directions with the questions, they're comfortable doing it too. You can get a technical conversation going. They won't lie and I say they can do X, the good candidates will quickly admit they haven't done X or Y, but their overall impression means not knowing X doesn't matter. The trouble is I've had so many candidates who are on the wrong end of the Dunning–Kruger effect.
[Minor grammar edits]
But I still prefer that to those stupid algo questions.
At a different company, I literally had an entire dev team sit on the opposite side of a table in a board room and play stump the programmer. After the interview I left hating the company regardless of the job offer.
Testing coding abilities should only be done in the context of finding out if I can trust the resume (which should be relatively minimal), or if the candidate is looking for their first job (again, minimal as they won't have experience).
The rest should be possible to analyse TALKING to the candidate or doing some join design session...
Good luck making people spending more than 4 hours in a selection process if you're not Google (and even so)... Seems like a great way of massively discarding anyone that's busy...
When a recent position asked for a 10-12 hour homework, said I cap them at 2-4 hours and quoted my daily rate. ;-)
This is still far cheaper than the costs of hiring the wrong candidate.
That said, I would also reject homework assignments, as long as I can afford it. If they want it, companies should pay for it.
Multistage interviews are also a big turn off. Imagine going on a date, and your date saying "I just need to try yet another date to see if I like you" multiple times.
The bottom line is you can't instantly validate how someone will perform over the course of a year or more. It takes time. But nobody wants to spend time so...
Also it shouldn't be the first stage of the interview process. That's just taking a piss. For this job it was after a phone interview, and with a scheduled face-to-face interview (where we discussed the code I'd written).
It's lower-status, more degrading and has worser odds than cold-call telemarketers trying to sell you some new credit card.
By the way, when a telemarketer tricks you into answering his call, you don't subject him to a 3-days intensive rectal exam ("homework") just for the remote possibility of getting a new credit card, which at the end you still don't buy anyways on some made-up pretense.
So developers agree to way worse level of abuse and degradation than the lowliest sales guy.
At least 5 years runway, entirely on your own "payroll". If you're actually competent, that's more than enough time to pick any skill (possibly starting your own business rather than look for a job using it), if you can do 12 hours per day by your own choosing on whatever you see fit.
Don't have those 5 years but only 1-2 or, god forbid, have nothing more than a few months on top of a large amount of credit you have to pay back.... and you're in the 99% percent of desperate and destitute crowd who will submit to anything in exchange for a bowl of soup (guarantee they never break out of slavery).
Problem with the solution is that it's a vicious circle: if you don't already have those 5 years of savings, chances are you're never going to save them anyways.
There is a lot of talk on HN about the best way to do IT recruitment, but whatever method is close it really shouldn't take more than a few hours of face-to-face contact and some legitimate tests. With HR doing due diligence by checking references etc after an offer is made.
The first time I encountered this I was told that I would have to do a 8-12 hour coding challenger over the weekend. They wanted me to come up with a way of proving someone has watched a training video without skipping or faking an API call - something they admitted in their interview that their system was lacking i.e. free work for them. I told the recruiter that I would not be doing any such thing. I have young children and my weekend time is precious. If you can't validate my skills from my a few interviews, my detailed CV, plus HR doing due-diligence, then it makes me wonder about the organisational culture of your company.
There is such a low barrier to entry for our industry that virtually anyone can can call themselves a software developer after a week of coding exercises, compared to other engineering disciplines which have a very high barrier (mostly in requiring a degree and often difficult certification like the PE). What's worse is that even computer science majors aren't guaranteed to be an effective hire. This is NOT a new problem[1].
Ultimately I would argue that the risk is higher, for a company, of hiring an incapable engineer, than it is for an applicant being hired (and paid) as an incapable engineer.
Requiring an applicant to demonstrate capability is smart. Where we draw the line for how much needs to be demonstrated is what is being questioned here.
[1] https://blog.codinghorror.com/why-cant-programmers-program/
I asked a friend who worked for years at a large public company if he would have passed the company's interview process at the time that he left. His answer: "most certainly not!". I think his answer is astonishing. Here is a guy who is considered a top notch engineer at his company, assigned to help the company screen for similar top notch engineers and yet, the established process would specifically fail a good percentage of high-quality engineers of the type that already work at the company.
I often wondered how sustainable that was. Will be interesting to see if they keep that strategy long-term.
For more senior roles, majority still rely on pedigree (previous employer, resume etc), and those who try to be more objective hand out a take-home which ends up frustrating both sides. Candidates have to spend a lot of time and effort and the hiring manager/devs have to setup/run/review.
This is where the industry needs to make the process easy, and operationally objective and efficient for the initial screen: An hour or 2 spent by candidate on a take home problem, which can be reviewed in under 5-10 minutes by the hiring manager. (This disparity in time is because the ratio of interviewed to screened is roughly that.)
(Disclosure: I work at hackerrank, and build the take-home equivalent of a coding test.)
I think there are a couple of solutions to this; 1) Instead of unpaid sample projects, companies could offer paid trial periods where you work on real problems and get paid, with the option to continue as a full time employee after the trial. 2) Instead of doing tens of sample projects or exercises, one for each company you apply to, you could be evaluated one time by a third party and that evaluation could substitute for technical interviews done by the companies themselves. It would save companies the cost of evaluating candidates projects, and it would save candidates the cost of proving themselves time and time again. The evaluation could potentially be handled by different entities; tech recruiters, certification bodies, or standardized tests. This may be what companies like HackerRank are attempting, but they don't really seem to follow through with job offers.
As it is, I'm trying to find a solution to my own job search. If anyone has any recommendations on how to find a job without burning out, I'd love to hear them.
2. A homework
3. An onsite with 5 rounds
Seriously, what does that additional tree traversal round prove?
1) because meaningfully large swaths of applicants are too stupid to fizzbuzz
2) because passing fizzbuzz is not the same as writing software
3) because teams are comprised of many people in many roles and just because team/person X likes you, doesn't mean you'll be a good aggregate fit.
2. Asking them to write fizzbuzz is exactly the same as asking them to write a method that does anything else, unless if they're aware of it and already knew how to do it, but of course you're asking them other questions as well?
3. Agreed, but it's impossible to match a new candidate with everyone in the company. If they're going to be working closely with person X then have that person sit in the interview for 15-30 minutes and see if they can work together on a problem.
1) I don't get fizzbuzz in first coding rounds anymore. It's usually graphs
2) Ok. I write software. Your whole team can go take a look at my software. Done coding?
3) Ok, so the last on-site round should be meet and greet then? Because I already submitted my code before. You guys all should have looked at the code and agreed to bring me on-site and just meet me. Why ask me to traverse trees again?
Hiring manager: Hey recruitment team, I've read articles saying <interview type A> doesn't work very well, we should try <interview type B>. <Interview type B> solves <concern X>
Recruitment team: Sure thing, we'll start with a parallel run so we can see how <interview type A> results compare with <interview type B>
Hiring manager: We've run both for a few months now, should we stop doing <interview type A> ?
Recruitment team: Some hiring managers still like it, as it addresses <concern Z>. We don't see it as our job to second-guess them. You can spend your time and political capital convincing every other hiring manager, if you like?
Hiring manager: I'm awfully busy with things that are a better use of my time.
Its probably one of the few parts of the process that actually makes sense, but the fact that people rarely agree belies the deeper problems with tech interviews.
The selection criteria is way too subjective and the bar at so many companies set artificially high.
In my experience your best bet is still to know someone on the inside.
Though even that doesn't always work. A former intern of mine was rejected for a full time job at the company because he was "shy". Mind you this was someone who built something still running in production today.
I do 3 or more interviews a week and the process seems like complete nonsense. (but its my job so I do what Im asked)
I've erred on the wrong side going the other way, recall one interview on a team where I thought something was off with the candidate because they were so abrasive and then they got into a shouting match with one of my teammates (we thought they were fighting in the conference room), but when it came down to it I didn't want to be the only no and reject them. That was my fault because they turned out to really be a sociopath after a couple of months.
The problem were the employers, who just treated TripleByte approval as an add-on, instead of a substitute.
One employer said I'd also have to do a take-home assignment (which was, I kid you not, recreate React) and then a 4-5 hour onsite. A few weeks later, that employer contacted me to see if I had completed the assignment. I told him I accepted another job. (Of course I didn't even bother starting the assignment)
Truth is many of these 'assignments' will never be looked at by a person that can actually derive good insights from the submitted work. Most of the superficial treatment they will receive could have been established in a five minute phone interview.
After reviewing the requirements I quickly determined that it would take me days to build an application that I felt would effectively WOW them. I told that to the recruiter and bowed out. Being already employed I looked at the opportunity cost and it was too high. Sure they paid more and are a bigger company, but that is A LOT of work. I rather do almost anything than unpaid programming on an application that will be thrown away in a few days.
If I were hiring, the "lab" portion of the interview would be simple. For mid to senior level developer:
1. Write a small inefficient application that requires some indexes, SQL optimization, and some code optimization. Very, very small. Very, very easy for a reasonable developer to fix. Afterwards I'd have a conversation about other general optimization things like Redis, Beanstalkd etc... because we'll optimization is important, especially to me.
2. A small class with very poor cyclomatic complexity. Clean it up. Because clean code is important.
3. A small insecure application. Things like not using bind parameters in queries, not cleaning inputs, XSS etc.. Secure it.
4. Show me something you've done. Personal or professional. I would then ask questions and it would be an open conversation. Because I want to see how much you enjoy programming and how we'll you can speak about it.
I think these 3 items, combined with the other interview questions, their experience, and resume would give me enough to go off. Ideally, 1, 2 & 3 can be done in under an hour. I think that is reasonable for pre-screaning an applicant. I certainly wouldn't send me running, but I am also writing the test so who knows...
Why would I want to look through 10 or so applicants full fledged application? God no. I have better things to do. I guess for some companies its a way of weeding people out, but these large tests are akin to using DDT to remove weeds.
Maybe paying people for the time if you end up saying "no" is a nice compromise?
If you are potentially serious about hiring the candidate and they are serious about potentially working for you, doing some "homework" does not seem like a major amount of effort in the grand scheme of things to me.
Guess which socioeconomic groups it tends to disadvantage most?
As a hiring manager (Director), I've long maintained the entire hiring process in tech is broken, and this movement toward requiring an arduous amount of unpaid personal time on "homework" is tantamount to abuse. Respect is a two-way street in the hiring process, and it starts with me showing them that respect for their time and obligations.
"we'd like you to just whiteboard some pseudo code for this problem that one of our senior engineers solved recently"
"was he stood in front of multiple people, timed and drawing on a whiteboard when he solved it?"
"no, but we're just wanting to see how you'd approach the problem"
"well, I'd probably do what he did which is Google some stuff, read some stuff, have time to process it, then write some code and then probably restart as I figure out the direction I want, probably bounce an idea off a colleague or two to make sure I'm not just in crazy tunnel vision land and then I might have the makings of a solution. Shall I whiteboard that flow diagram for you?"
interview ended abruptly at that point
Probably it helped that we were in a smallish college town w/o a ton of other tech, & this was a while ago when the tech job market wasn't as crazy hot as it is right now.
(Also, I want to emphasize, it was a challenging problem, but it wasn't a lot of code at the end of the day. If you knew what you wanted to do, you could knock it out in a couple of hours. But you had 3 days to mull it over or try different approaches. We were not asking you to build an app or something similar involving a ton of work.)
I'm sympathetic to the fact that it's a big ask, but I really truly feel like it gave me a much better sense of candidates than just an interview & a resume (what we do at the company I work at now).
It was also a fun problem to solve, and people would tend to continue to refine their solutions (to be better and faster) competitively even after they'd joined.
FWIW.
They make it a point of ego / pride that they have such a long, arduous process. They've even mentioned the occasional candidate who asks about billable hours for an 8+ hour code problem, saying "Who the hell does that guy think he is?" Of course, they were shocked and appalled when I asked who they thought they were to ask someone to spend 8+ hours of their time on an code test before even having so much as a phone screen to determine compatibility.
As it were, this was a company in the contracting space so I assume this is an intentional filter to weed out all but the most compliant individuals. If you're not willing to bend over backwards, they're not interested.
I know that games developer ThreeRings did that. Once you got far enough along, they'd give you their games toolkit and ask you to build a simple, original game in, I think, 20 hours, for which they'd pay you a decent rate.
I'm not sure I'd do that today, as it can exacerbate existing systemic bias issues, and cause you to miss out on good candidates who have lives where it's hard to free up 20 hours on short notice.
Plus, I think there are cheaper ways to get that data. I now structure my interviews as joint work sessions. One round is a couple hours of pair programming; the other is a couple of hours of joint design and discussion with the team. The goal of both is to see people doing something pretty close to everyday work. I think I can get a much better feel for somebody's skills in a few hours of collaboration than 3 days of remote work with a big-bang delivery at the end.
Seriously, this trend can go fuck itself... the last time an recruiter told me I had to write 100 words about 12 different skills I told her no, and hung up - only to send her a ranty email about how stupid the idea is.
The two worst developers I've ever had in my team both aced those interviews. How? Their colleagues did most of the work for them. It worked out great for me, it cleared out the dead wood and punished some faker at another company :)
I'm very quickly realizing it wasn't that great and in fact, comes off as lazy from the interviewer perspective. There are several "coding interview websites" where you can ask the question, give the candidate a terminal to code, provide a list of available languages, and even have unit tests from the interviewer side to make sure the solution is correct. We've completely abstracted out the interaction and questioning of candidates and it has become a binary "did they get it right or not" type of process. Sure, there's follow up interviews and such but what have you gained from this process that couldn't be asked over the phone?
Worse yet, if you aren't using these tools and simply letting the candidate email you a project (or however they choose to deliver it to you), it can quickly become a time sink as you spend it trying to understand the solution (i.e. why did they interpret the meaning of "API" as a REST service? What is this obscure library they're using? etc.)
No thanks.
I don’t know if I’m going to be able to go through all this bullshit the next time I’m looking for a job.
I have a friend who is a pharmacist and I asked her what a typical interview is like for her. She said interviewers will typically ask something like “tell me about a difficult situation you had to deal with at work?”.
No homework, whiteboards, group interviews, or stupid puzzles. No bullshit.
That's just another form of bullshit. I'd much rather write a for loop than try to figure out if the interviewer wants an honest answer or a humblebrag.
I've been going back and forth on how best to handle this newest trend. I don't think it's a good approach to just refuse to do them. But accepting the challenge and then not following through is even worse. So a mutual understanding of the deliverables and level of effort is required, otherwise job hunting approaches second job status.
My current stance is to first negotiate the code challenge until it's fully in your wheelhouse. If you are unable to do this, politely decline to do the code challenge, citing time constraints. One job wanted me to do a basic web app wholly in vanilla Javascript with no libraries. That's a challenge I should have just declined.
Second, refuse to accept time limitations. No company that's not paying you should be allowed to monopolize your time. One job had a hard 20 day limit and the task was both outside my wheelhouse and expected to make a measurable delta in their production application. I definitely should have declined that one.
But if a company is willing to work with you rather than subject you to an inhumanly-inflexible meat grinder, that's the one I want to work for.
Candidate is also evaluating whether company is worth working for.
if you give a take home assignment and don't give me a feedback on stuff I could do better, i would consider the whole thing a waste of time. So if you don't offer that's ok. But let the guy learn a thing or two.
Many people will agree that yc interview process is rigorous but it's also satisfying as you get to learn a lot through out.
Cue the music (sung to the tune of Eminem's 8 Mile Road, see https://bit.ly/2iI8tG2) and the scene of a nerd sweating in front of the whiteboard as the interviewers nod their heads to the beat:
Sometimes I just feel like, quittin', I still might Why do I put up this fight, why do I still write (code)?
Sometimes I just hate life, something ain't right Hit the brake lights, case of the stage fright Drawing a blank like
It ain't my fault, great big eyeballs My insides crawl, FizzBuzz I scrawl And I clam up, I just slam shut
Gonna travel this road Gonna show 'em my code My Big-O is so dope the universe will implode (as our hero proudly writes = O(1/n^2) on the board)
etc. Someone can make a viral video; I'm too busy practicing code problems.
People like me are highly employable/contractable. So, the way an interview works is very simple: you got lucky enough for me to take note of the fact that you are looking for somebody with my skill-set and I've determined that you are worth talking to. Fantastic, because I ignore all HR & recruiter spam so you probably found me through some mutual contact that I trust. Great, now we meet and my expectation is for you to decide to hire me on the spot and be extremely enthusiastic about it and get in a mode where you are selling the job to me as something that I want. We both know that nobody else similarly qualified is just going to walk in and take the job. If so anyway, my advice is to hire them on the spot. No hard feelings. So, very simple choice. Either we like each other or we move on. You don't get to ponder your decision for very long. The words "home work exercise" typically don't feature in such a conversation.
A friend recently spent a week building a chat server with a very specific telnet/ssh interface as a "code test" for a video game company. I helped him test it and it was really solid. He walked the interviewer through the setup and how everything worked and then never heard back from them. I suspect that they simply used his code in their next project. What's stopping them from handing over his code to the next round of applicants with the assignment of adding all the features they need? They could post a high salary and offer great benefits, but never actually hire anyone.
We, as developers, simply can't be giving away free work. We're all aware of how opportunistic business folks can be. I refuse to do "homework assignments" (unpaid work) and so should you.
As an interviewee, it gives you a glimpse of what work will be like once you come on board. Are the requirements solid or loose? Is testing required or even mentioned. These and other things can give you insight into how the business interacts with engineering.
As an interviewer it's hard to take resumes and unfortunately even GitHub profiles at face value. I've personally had code taken from work put up on a colleagues personal GitHub. The coding task is also not a sure-fire way, as I've seen those taken from online tutorials. The real magic is in the tasks code review.
I'm not saying this is how it works everywhere, but given the task matches the deliverables asked and we make it to the code review stage, it becomes less of a tech interview and more of a glimpse into the social skills of the developer. PR criticism is a daily occurrence and getting an opportunity to test these waters prior to fully committing to onboarding a developer.
- Write a game engine
- Answer 5 programming problems in the span of 2h20m
- Do 3 coding exercises over the span of 12 hours. (That was pretty brutal .. as that you have to build everything up yourself)
- Write a REST service and business logic
- Write a rules engine with depending rules and type.
Worst part of all of this:
When you're getting evaluated you're up for being nitpicked over spelling mistakes in the documentation.
If I am going to get rejected, then have the professional courtesy to give me constructive feedback on the code. In the future, I will ask for this up front before taking any more tests.
It looked like it might be a devops position with a smallish family video site. After an NDA (?!!) and some emails, no interview, they set me up with an obscure chat client with a guy for a tryout. He had me set up some services on a new instance, apache, php, firewall, etc, following a checklist they gave. He wouldn't answer any questions about the company. I did this quickly and that was the end of it. In the end, they said no thanks, but did not give any feedback. They paid me for a day of work on Elance :-/ $50 for a couple hours of work.
The bad news is the whole process was super shrouded and I to this day do not know why. The NDA forbade revealing any contacts, who the site was, or anything else. Nobody was willing to give me their name or answer any questions. I guess the vanilla family site could have been a sister/front for a gambling or pr0n or offshore hedgefund company, or ... what? Why bother?
Then these geniuses decided to profit and "flatten the world" as they call it. Just so they can make a buck. https://www.hackerrank.com/aboutus
The only thing that is going to get you is a flat brain. Unable to think creatively, unable to respond to uncertainty but very very good at memorizing solutions to problems.
And all this is based on this ignorant viewpoint that you can observe something or a product for a short period of time and somehow ascertain anything which you can use to form a useful opinion of a candidates future performance. Bullshit!!!
No HR department should be allowed to ask any candidate to do any work without compensation. It is work since the HR department is using this information to make their decision therefore protecting or enhancing their companies performance. I don't know how this is not related work to a companies success.
I am sick of it. I have started to send templated emails in response to them which explain i do not work for free. I can share personal code from past projects or my github.
A few months ago a PR representative even called me once to make sure i was not going to sue when part of the application process was a take home assignment which was asking me to implement a new feature for their actual product. What a sham.
When i said to her "You mean to tell me that over the past years when you claimed that this process has been the same, no interviewer has derived any benefit from free work of past candidates." She didn't even know how to respond.
The more candidates that do not put up with this behavior, the faster it will stop and hopefully move onto something more agreeable by both parties.
For me personally, I am not so much for or against 'homework', as would rather see it used correctly. For example, I want to know how it will be evaluated, no tricks. I also think there is a quickly diminishing return on the length of the assignment. Up to ~10 hours may be OK, beyond that seems like a waste.
On the other hand, it's also a way to see who's serious. If you are serious about getting a job, it may take some investment. Also, the most glamorous companies can always demand a higher investment from applicants.
1. https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=work...
I think these tactics are a lazy way to interview. If you truly want to build a great company, you need to spend significant time with someone to understand 1) can they do this job, and 2) are they a fit with the culture, and 3) are they passionate about the mission. There are no shortcuts, yet so many companies try to hack the process.
Everyone is busy and looking for ways to steal time but this seems to be one of the most short-sighted. The best companies don't look at labor as just an input.
"let me know show you some of the software I've developed on my own - I'd me happy to tell you how it works, the tech involved and the patterns used ..yada yada"
Corollary : if you apply for a job or walk into a tech interview and can't talk about these things, its a red flag for the interviewer.
PS. Now well on in my career, and having started my own software business after leaving a bigco. I'm back to a very similar thing: if I go to a meetup / conference / sales call I better have some code I'm will to show and discuss ..and highlight what I consider my talents. (fwiw : it feels great to be back in this mode)
edit: also : asking an candidate to do the kind of homework/coding is total BS. I wouldn't do it. But I would give them the line above :)
They may have built software on their own, but it would all be internal, proprietary stuff that they can't demo, and you have no way of verifying.
This is especially possible given that a lot of work includes open source code, publicly available APIs, SDK services etc. In interviews I've conducted, when a candidate mentions these its an opportunity to dive into more detail on their expertise ..and again discuss contrived examples.
In addition, probationary periods are a way to guard against complete mismatches.
So then actually I could not do anything about it until 5 in the afternoon the next day, I had problems with environment setup (hadn't needed this particular stack on my new machine so I hadn't set up and things had changed) so it took 3 hours. then I had to do dinner with the family, get kids to sleep. I didn't have the time available in my weekend to do a 3-4 hour assignment that also required 3 hours to get ready to work.
I sent off an email to the recruiter and said what happened and I wasn't really interested in anything similar in the future.
That's your main mistake, admitting that. I'm sure they were looking for someone with five more years of experience than that.
You should have been writing apps in a text editor waiting for the app store to come into existence, you know. I'm sure they picked someone with at least that much experience.
In the past, I would sometimes accept the homework assignment. Now I refuse to do those (as opposed timed assignments where you get set amount of time, for example 2h, in which you are supposed to complete the assignment).
If the homework deadline is few days you can practically be sure that there will be candidates that will spend all that time perfecting their solution. What is supposed to take "2-4 hours of effort" in the words of the interviewer is actually the entire duration from the moment your competitors got their assignments until the deadline.
There are other reasons to refuse:
- you can spend that time working to find other offers,
- the interview is supposed to be both ways, they want to learn who you are but you are also supposed to learn who they are. Home assignments give practically nil information about your prospective employer.
- there are other ways to test the candidate with probably same or better results. Choosing a way which minimizes the effort on the part of employer but maximizes the effort of candidate shows lack of respect for the candidate.
- Home assignments are invitation for cheating. I can assume there are candidates that will take advice from others on how to solve the problems they are given so this is hardly proof of their ability. I refuse to take a test where I will be punished for my honesty.
- Even though the assignments tend to be time consuming they are typically hardly challenging. This is typically some mundane application and not really a test of your ability. My ability is to solve complex problems, organize complex logic, produce extremely reliable or efficient applications, draw from wealth of my experience to propose and discuss solutions. Yet another Pet shop written in technology of employer's choice is hardly any test. If you want to know if I can program at all you can learn it in few seconds of conversation with me, you don't need to force me to do 2 days of programming for this.
Assignment was to develop an application which tested the candidate's skills to access web api, browse through documentation, do proper authentication, parse api results and finally the most important skill to design an algorithm to compute desired output depending on the question asked. It was doable within 3 hrs for experienced engineer. Not many succeeded in this task but those who were able to do it were real good developers.
But then I realized that the occasional two days of unpaid homework is still way better than being required to have a college degree or professional certification - unlike nearly every other profession that pays over $100k.
I'll take it. For now.
When you get called to audition, you spend a ton of time preparing at in your "free time".
If you want to apply as a (senior) sw engineer, it makes sense to have an idea about the depth of knowledge and/or the engineering approach of the candidate.
Oldie but goodie: https://blog.codinghorror.com/why-cant-programmers-program/
* It can't be taken as real work at all. Something like building a game is clearly not "real" for a SaaS company and still conveys lots of value
* It should be discussed in person as the in-person technical interview (don't come in an whiteboard in addition to the outside work)
* It is provided only after a phone screen and it is discussed on the screen.
* There are clear objectives and "bonuses"
* The language of choice can be used
With this, I think that it provides value to the interview process and helps the interviewee as well by simplifying the in-person technical components of the interview.
I know noone forces me to do it, but back then I really had high hopes for that job. Nowdays the maximum I'm willing to spend with an entry app is two to four days but deep in my heart I still find it a bit too much, but I still prefer this over live coding sessions in front of a bunch of random devs.
The worst is that when they ask you to take three or five days off and work with the future team to see how you fit in (not for free though). I'm not too enthusiastic about it, but I already did it twice.
The latter one had a fairly complex entry task which took me three full days and I had six interviews after that - during the seventh I gave up and told them we are not looking for each other here.
- select your candidates that maintain a public profile. You should get a broad idea about the skills and interest of that person
- have an informal interview to see if they fit your company culture and values
- then have them work on a small real project for less money
I'm a dev and happy to work on such a "taster" project. It's a win-win: I'm not working for free, the company gets a realistic picture of how I work and how I fit into the company.
This works naturally best for hiring contractors, but even then I don't understand why not more companies do this.
More pathetic are only companies which have only one question at the beginning (without any further contact) "how much do you want to earn" and after that there is no negotiation. Sometimes I feel like I'm drowning in an endless sea of candidates. Oh, wait, no, the same companies publish the same job ads over and over for years. So maybe not.
Would you want to work in a world where all programmers have to be trained, licensed and regulated with the same rigor as surgeons?
Surely the fact that programming is one of the few 'white collar' jobs where someone with no education and no relevant work experience can get hired based purely on the fact that they've taught themselves the necessary skills and can do the job is a net good thing.
Uh, yes...
There are too many terrible developers out there.
Either that, or they're blatant featherbedders who aren't bothering to disclose their interests.
In neither case do they belong on this site.
If you look at other engineering disciplines there are formal structured approaches to education and testing leading to chartered engineers etc. We could really do with the same.
At the moment we have certifications which merely prove that the individual can pass an exam. I can routinely get near 100% on these sorts of test without knowing the subject or having any experience.
I can talk about bad recruitment processes for hours. I just want to meet competent recruiters, and managers - just like they want to find a competent candidate.
Yes. Please, God, yes.
But intentionally blew off the 2nd quiz (thus the interview) before starting the assignment due to a family medical emergency (I thought, WTF am I doing, there's more to life!)
Nowadays I find it's either: a. multiple quizzes or HW assigment b. a presentation (that takes 4-8hrs to make anyway). c. and that github was a source of code eval, not anymore! ...cause everyone forks everyone's stuff (that's ok, by design).
Learn three different frameworks, unlisted in the job description, and extend a github repo, suspiciously similar to their production codebase.
That they insisted what was easily a 10-hour assignment could be done in 30 minutes "if you're already familiar", made it seem like they were offloading the learning-on-the-job to before-getting-an-offer.
Cunning, but ultimately a sign not to sign.
The company I currently work at has very few employees, and each new hire is a big cost. If a new employee does not work out, that can be a major dent in the budget. We still haven't figured out a great way to interview, and we are open to new ideas, as we have had hires who looked great on resume/interview who did not end up working out.
They have already committed that day to this interview, there is no abuse - show us you can solve problems and not give up after a couple of Google searches, like a good half of programmer-aspirants do.
After each attempt we said no that's not right, and then at the end we were asked if we were going to use his code in our platform and if he should be paid.
Felt kinda bad for him, clearly didn't pass the interview test.
This sounds like a good idea to try. Much less time consuming than having to create meaningfully things from scratch. Also more relevant as you don't usually need to write code in front of others, but you might need to review existing code.
I don't ever hear of structural engineers or medical doctors having to solve puzzle to get past their interview.
Out if curiosity, what kind of assignments are you being given?
Slides https://slides.com/scottconnerly/2questioncodequiz
Video: https://youtu.be/x8vmNXULbp4
Phase 1: 30min non-technical phone screen. Usually we would have three people on our side of the call, but only one of us would do most of the talking. The goals were simply to describe the company and role to the candidate, gauge the candidate’s interest and ability to communicate, and allow the candidate to ask basic questions. We would discuss and decide whether to advance the candidate immediately afterward and usually notify the candidate on the same day.
Phase 2: in-person code review, typically 1hr. We’d give the candidate the option of providing us with a (working) code sample from something the candidate had written previously, or of coding up a solution to a problem faced by our actual product. While I was there we had a good mix of candidates picking each option. The “homework” problems were always drawn from the product, so they were real, but we also kept them small, so that an average candidate could have a working solution in an hour or two. And we’d give the candidate a week with no other constraints. Candidates could use any language, could research solutions however they liked, and could email us any clarifying questions they had. Whichever option the candidate picked, the goal was just to get our eyes on a dozen sample the candidate had written and could be expected to understand. At the on-site, we’d have the candidate explain the problem in the candidate’s own words, then walk us through the solution. The goals were to determine how well the candidate understood the problem and how comfortable the candidate was with explaining technical decisions. It didn't matter to us if the code ran or even attempted an optimal solution; we advanced some candidates whose code samples did neither, but who were able to talk us through what they had tried to do and explain why it didn't work.
Phase 3: working interview. We hired candidates for a day, paying $1,200–$1,600 for the day, depending on the role. The candidate would pair with a member of the team and work on some aspect of the product. The task was selected to be representative of the work the role required, but also simple enough that could be completed in a single day. At the end of the interview, the candidate got to ship the code to production (for which purpose we had a big, important-looking, physical red button). The goals were to determine how well the candidate functions in a working environment, how readily the candidate asks for help or conducts research, how the candidate responds to frustration, and how interesting and engaging the candidate finds the work.
It was quite a time consuming process on our end, but it was extremely effective at finding excellent candidates. We never made a bad hire the whole time I was there.
When i interviewed for a backend development position, the task was to set up flask with two routes, connect to a dB and retrieve a couple of rows when user visited the routes.
It was not hard even though I never worked with flask before, and it wasn't supposed to be hard either. It was a test if I had basic programming skills and that was it.
When I found myself out of work last year, I interviewed for a big auto company - their interview process involved an online coding session. They gave me a spec for an assignment and asked me to complete it in 20 minutes - I could pick Java or Python. I picked Java, and got the spec - it was ridiculously simple. I can't remember the exact wording, but it was something like, "write a function that will print out the text value of a number in the range 0-4, given the numeric value (so if you got an argument of 3, you'd return 'THREE')". Well, in my haste to complete the assignment in the generous 20 minutes they gave me, I... didn't read the specification closely enough. I started writing a program that took in a _list_ of numbers on the command line and output their text values. It took about 10 minutes to code it and sanity test it - then I thought to double-check the description and, "shit, I misread it". So now, with 10 minutes remaining, I'm thinking, "if I change this to fit the description, they're going to see that it took me 15 minutes to come up with a pathetic function that should have taken me 2 minutes." So, I started refactoring the whole program to _contain_ the function that they asked for, but call it from a main routine that would parse the input, call the function, and write out the results. With 19 minutes on the clock, I submitted my abomination. The recruiter was kind enough to call me back and tell me that they had decided they had "better qualified candidates in the pipeline". Oh, well...
Probably the ones that read the spec and delivered what was required. If you're going to gloss over something trivial, what does that say about your attention to detail?
I'm willing to give up an evening for a job that appeals to me and that I have a decent chance at landing. I'm not willing to give up all my weekday free time for a week or stay up until 3am. And I won't do either if you're not willing to first invest 15 minutes to make sure there's sufficient mutual interest to be worth my investment of hours.
If you don't respect my time before hiring, I assume you won't respect it after either.
That was what stopped us doing it, and started looking at reasonable tests that you can't prepare for, and can't spend more than the allotted time on. So that means we weave the testing into a 1-2 hours interview, and everyone has the same chance.
(we also document our process so you can see what you're getting into before you start)
We do require candidates to use version control and (though we don't tell them so) we review individual commits to understand the candidate's approach and how they use version control, and commit timestamps to see how long implementation took them.
And the volume of candidates who pass both resume review and phone screen in our process is low enough that we're not usually comparing candidates to one another; it's "can you do the job or not". So relative advantages aren't such a big deal.
I did an iq test that was not timed and got 10 points higher, which is quite significant and also much more how real life problem solving works as a programmer. I'm never under any time pressure in my job.
Setting up a web server is also a different skill than backend programming -- many experienced back-end programmers have never needed to spin up a new server for their job.
Ill give you an example of a task that I got in my 2nd/3rd year of employment.
Came into work and was presented with an exotic (for the time) piece of hardware (A0 digitizer) that cost 3x my salary to be asked to.
1 Connect it up and get it talking to a PDP
2 Research and Write the code to allow the kit to interrupt the os and transfer the data to a second program (this is on RT11 which only allowed 1 process normally).
3 Work how to initialize and calibrate the kit
4 Was asked to go and chat with one of the senior engineers and work out how to use the kit to digitize the results from his experiment to produce a 3d representation of the droplet cloud - z axis was done by using a camera with a very tight focal plane
Another way to look at this is, are you more concerned with testing a person's end-to-end or development skills? There's no right answer, but you have to realize that testing one comes at the expense of the other. If your company often uses brand new development stacks, it might be important to test a candidate's ability to do exactly that, but if your company has a framework in place that everyone uses, then it's probably best to focus on a candidates pure dev skills.
The worst one was for Optimizely, who asked me to write an entire website (front-end using React, backend in Python) which will display a list of products from a database, including full tests. I determined that boilerplate and configuration would take me about 5 hours, with maybe another 5-6 hours of actual coding. They even offered to pay me something like $15/hour (literally nothing) to do this. I said no.
Bells and whistles are likely to have the opposite effect on me FWIW. We've got enough work to do around here without inventing more work for yourself. 'Keep It Simple Stupid' as it were.
This is approaching simple lazyness / entitlement.
The free work is the issue. Pay me for a day to do a project to validate my abilities and we're golden. I've done that at a few places, and it's a great way to go.