Developer hiring and the market for lemons
danluu.com
danluu.com
And yet, I've known more than a handful of developers who are really solid, great people to work with, and I'd like to work with again. None of them have ever blogged, podcasting, presented at conferences nor run a public repository on github. Yet by many definitions of 'great', they fit the bill, but not the sort of 'great' that would mean companies would just 'know' of their greatness, and go out of their way to hire/recruit them.
Not really.
What Joel and Dan are both talking about is that great developers tend to have coworkers who know they're great and will happily hire them at other companies.
I'm certainly not a "public" developer. My most popular repo on GitHub has maybe 10 stars. But I do have a handful of people who I've worked with in the past who will readily hire me at their new companies.
Referrals are how most hiring works, and it's the predominant way that companies recognize "great" developers even though said developers are not public.
Why? That's 3 companies where someone should be willing to give you a strong referral, not to mention that really your network should include second-degree referrals as well.
I'm not sure what you mean by "second-degree referrals".
Reacting more perhaps to this general notion (or Joel's notion?)
Exactly! People they know tell them about jobs and then vouch for them.
> It would feel extremely awkward to now, years later, send them an email
If you ever have a reason to send that email I hope you do. Most people enjoy a chance to help someone out and they'll often get a bonus for it. I've gotten emails from people I didn't actually know and just sent them on to the recruiting team. For people I did know or had a trusted recommendation for I'd follow up with the hiring manager directly.
Anyone that I worked with (even 20 years ago) and who was a good colleague then would be more than welcome to ask me for help in a job search. I'd think of it as totally normal and something that I'm happy to do.
You are definitely an outlier in the industry.
By far the most common way to hire people is through referrals.
Whenever I am looking for a new job, I send out emails to my network saying I'm looking for a new opportunity and to let me know if they know anything interesting. It's a win-win for everyone.
Source?
> Whenever I am looking for a new job
So a sample of one against a sample of one is "definitely"? :-)
I have yet to see anyone do that, and I'm connected to quite a few colleagues from when I was still employed, not just from the freelancer period.
I am, frankly, amazed that you have never had a former colleague approach you about a job opening they had or express that they're looking for a new job. That's literally one of the most basic uses of networking out there, not to mention that many companies have explicit cash bonuses for referring new hires.
Here's a recent survey showing that most hiring comes through networking: https://www.linkedin.com/pulse/new-survey-reveals-85-all-job...
This practise seemed alien to me as well... but it turns out that it really is a common and accepted practise.
Maybe you don't but a great many people do, myself included. If you did a great job, people remember that. Of the colleagues I've worked with in the past that I respected, I would have no hesitation in passing on leads to them or referring them to my current employer.
It can seem paradoxical that companies are saying that they can't find people to hire, and even with the tech boom there are developers who can't find jobs (at least ones that they are completely happy with), but this is because a huge percentage of CVs on desks are from people who have needed to take the time to send off job applications, and are often therefore representative of the not "great" developers.
But the notion that the best developers rarely go through the normal interview process is itself problematic and worth picking apart.
It's true: the developers with the best reputations don't get interviewed. They can get their pick of jobs just by networking.
But that's bad.
It's bad for multiple reasons:
* It creates an informal guild of developers --- not all of whom are actually good --- who can bank on career mobility regardless of performance. Those developers are often hard to motivate and always hard to retain. Some of them are toxic, but because it takes 1+ years for most companies to cough up a bad developer, they end up with resumes that are indistinguishable from the rest of the cool kids, so they get to "fail up".
* It means some of the best developers are never informing the hiring processes of companies, because they get to skip unreasonable candidate screens. This means the screens never get any better. If you're serious about screening candidates, you standardize your process and you have an ironclad nobody skips the process rule. But almost nobody has these rules, because they feel like they can't and retain access to the cool kids guild.
* It reinforces and amplifies stereotypes and privileges. It means it's super easy to get a job if you know the right people and look like Zuckerberg. It's part of the reason the demographics of our industry look this way.
So, apropos little else of this thread:
Don't let elite candidates skip your process. Standardize your process. If an elite candidate balks, then you've learned something about them: they'd rather your process suck and keep sucking than take any time to help fix it. That sounds to me like someone who isn't prepared to invest their time and energy in your team.
And why should they? If you are running them through some elaborate hazing ritual before they are a member of the team. If at any point during the hazing ritual, any unknown person can black-ball them, in what way do they owe you the courtesy of helping you improve your team?
The contract in this instance is not "I will help you out of the goodness of my heart" the contract is "I will help you for x hours a day in return for a wage of y"
Anyways, I appreciate where you're coming from, and agree that you want to work with "generally agreeable" people, and should hire the same.
In my opinion, it does not necessarily logically follow, that someone who is unwilling to prostrate themselves before a noxious process is not willing to invest in the team they join.
It is incumbent upon the hirer to respect the applicant as well as the inverse.
Now, if there are two different people who rank a candidate badly, or the other interviewers are meh, that might be taken more seriously.
I agree about the hazing ritual aspect of it though. I think it comes from engineers doing interviews who are just going through the motions without taking it seriously or trying to improve, which is what you get if everyone is supposed to do interviews.
> Amazon’s “bar raiser” interviewer is charged with keeping the interview bar high. They at- tend special training and will interview candidates outside their group in order to balance out the group itself. If one interview seems significantly harder and different, that’s most like- ly the bar raiser. This person has both significant experience with interviews and veto power in the hiring decision. You will meet with your recruiter at the end of the day.
I can confirm that for me, the interview was 4*45 minute whiteboard coding sessions. In two of the interviews it was 2-on-one.
Unlike Gayle's description however, I received no debrief or description of what aspect of the 4+ hour ordeal I was rejected on. (Not to mention the night before where there was a 5 hour social/mixer).
It seems that since her writing, they've adopted a no-feedback policy.
So all in, I spent a day "investing the effort in their team's interviewing techniques" but got nothing but a black hole in return and down a day's vacation.
I've got a lot of sympathy for the person who says "this interview process is a lot of work" and who, after 17 years coding in production, decides they don't have time to invest in helping a room full of 20 somethings improve their interview skills.
It probably sounds a lot more bitter than I mean it to. I've looked for work 3 times since '98 and found it pretty quickly each time. (One of those times was within a day of starting to look)
I do worry about ageism going forward. Fortunately my current place has a lot of those gandalf-looking fellas. I love working with they grey-beards of programming.
In fairness, I think my experience with Facebook's interview process was worse.
> Standardize your process. If an elite candidate balks, then you've learned something about them: they'd rather your process suck and keep sucking than take any time to help fix it. That sounds to me like someone who isn't prepared to invest their time and energy in your team.
... huh? A candidate has no power to change your process. They aren't employed by your company yet. Even if they do go through the process and become an employee, it probably won't be in the capacity of senior manager, which is the rank that has some power to change the process.
If you ask the candidate 'okay, what do you see as the problem with our process, what do you think should be changed?' I'm sure you'll get some answers. Granted, if a candidate replied 'I'm not going to tell you. Looks like your process will just have to keep sucking!' then you could reasonably conclude that grape was sour anyway. I would be surprised to actually encounter that reaction; have you ever encountered it?
I get why a calibration process is useful to the company, but it's not particularly useful to the person who pays the costs.
Before a new session or technical question gets added to your company's tech interview corpus, you require the team to dogfood it on in-house engineers. If the people you've already hired have poor feedback on the question/session, it may be in need of serious work or may just be something you shouldn't use.
they'd rather your process suck and keep sucking than
take any time to help fix it. That sounds to me like
someone who isn't prepared to invest their time and
energy in your team.
Gotta have a pretty good prize if you want people to gamble for it.
It could also be that your interview process is bad and you should change it.
Perhaps the elite candidate is a canary in a coal mine, and fixing your interview would help your less privileged interviewees, too?
Yes, exactly. After all, fancy new dev is here to help your company, right? Well, make his first job contributing his insight and expertise to the interview process.
Those of us who don't have days or weeks to waste interviewing do what we can to build a good team. Yeah, that often means allowing good engineers to fall through the cracks, but we're all optimizing for time & effort.
If that is the case, either the cool kids are wrong and your test is doing an excellent job of evaluating your candidates, or they are right and you should find a new test which they and other candidates will be able to go through.
This is arguable. Personally, of the local (London) developers I look up to, not one of them is in what we'd call here a prestigious job. They are still forced to apply to jobs as any other average candidate. I am referring to people I've worked with, and other with amazing github repos.
Despite all we've heard, I am forced to conclude that talent is mostly unrecognized in this industry.
Yeah, no. A thousand times no. The interview process isn't just you (the company) interviewing the developer, it's the developer interviewing your company. If I saw tons of red flags suggesting:
* a lack of knowledge about the software development process
* a lack of respect for the time of others, paid or unpaid
* a lack of standardized tooling and an unwillingness to use it
* a lack of flexibility (e.g. the inability to recognize multiple ways of accomplishing the same task or to go off script)
* a dependence upon non-managers to pick up after managers and administrative staff
* a surplus of bureaucracy or bikeshedding through which people exercise power
* kool-aid and enthusiasm instead of proper remuneration
I'd respectfully excuse myself from further contact and run the other way. New positions come with limited authority. Few good employees get excited about working their way up the respect ladder just to finally get the chance to tell everyone how inefficient they're being. And it certainly doesn't build those new team relationships to get hired and immediately turn around and scrap the recruiting process.
Yes, there are polite ways to approach someone about bad processes, but there are some people who don't like advice, even if it's polite. If they mention that they're clueless about hiring and that they want help, yeah, you can pitch in. But that's pretty rare in interviews, as it's not usually a positive factor in attracting talent.
Fixing broken code is one thing. That just planning with some T&M. Fixing a broken culture or business model is another. Usually that involves offending some old goat(s), playing politics, advocating for transitions, and in general stepping on lots of toes. That's not a software developer - that's a business consultant. There's a reason why those are generally not employees.
If you're wanting business advice, throw an extra zero on the end of the compensation and give them the actual authority to make changes. Then they'll be glad to give you all the unsolicited advice you can stand. But don't expect people interviewing for non-managerial positions to be able to (or want to) do your job for you. Some people enjoy that, but just as many hate it. And that preference has absolutely nothing to do with their skill or work ethic.
It's kind of obvious that no one really knows how the perfect way to select good software developers through the interview process. That's why we put up with interviews. But a bad interview can be sufficient reason to skip a potential employer.
"This model is certainly different from Joel’s model. Joel’s model assumes that “great” developers are sticky – that they stay at each job for a long time."
Is false. Joel simply did not say that. Joel actually said:
"..prospective employers recognize their greatness quickly, which means, basically, they get to work wherever they want, so they honestly don’t send out a lot of resumes or apply for a lot of jobs."
Great devs aren't on the market because they don't use the "market" to get their next job - not because the don't leave jobs very often.
Not that I am some great developer, but in multiple jobs over almost 20 years I have sent a blind resume exactly once. Even then, it was on a whim.
In the other cities of the world. There aren't that many places where you can work at in the area. Your network is limited and you still have to go through traditional interviews (supposing there are even companies to work for in the area).
But, in some ways, all-interview-no-hire is worse for candidates than no-interview. When you interview you're blowing at least one vacation day, and when the company is not serious about hiring it's a total waste.
But a lot of it is anecdote and faulty assumptions. I don't know of many other professions that have 8 hour oral exams that test the sum of all knowledge in the field with such a strong negative bias towards hiring. Or a profession that so strongly assumes ability is innate and cannot be trained.
It's outrageous.
It may be that coaching, pairing and code reviews are simply ineffective in teaching people. They are also very expensive, since they sap the productivity of the rest of the team, and make planning and sequencing less predictable.
Overall, in my experience it's a lot less costly to turn down a candidate that might be good than to take one on.
(FWIW, the main metric in our company's hiring after meeting a minimum bar of an online coding test (~15 minutes to solution) is pairing with them on a problem that can be solved in several different ways. Getting a key insight into how to solve the problem isn't the metric; it's communicating different approaches and seeing how much explanation is needed to get the implementation idea across and how fluent they are at turning the idea into code. It's 2 hours of working together; it is not 8 hours of trivia.)
I was very much a weak candidate when hired for my first programming job after university. No experience with shipping code or working on large code bases, virtually no experience with C++ or OpenGL/3D programming for an application where the core was doing 3D visualisation in C++, minimal relevant domain knowledge etc. etc. And I'd really like to think that not only did I get much better in the first 6 month I was there on a temp contract, but that I kept getting better in every aspect of the profession for the rest of the 3 years I was at that job. And even being on the other side of the 'table' I've seen people who could hardly code at all turn out perfectly acceptable code 6 month later.
It doesn't even take "extensive coaching and pairing", just a lot of nudging and pointing in the right direction, some well timed pieces of good advice and the occasional bollocking when they're being just a bit extra dense.
When interviewing a junior, ideally they'll have been programming since before college and are actually intermediate devs in disguise. But when actually hiring a confirmed junior, you look for a spark of intelligence but also go in on the understanding that (a) their potential is unknown, and (b) there are only so many juniors you can allocate across teams and maintain productivity. Juniors are a speculative investment.
Classical musicians.
Of course there will be relatively few areas where early involvement is even considered in a candidate. How many 9 year old lawyers do you know?
Of course these arbitrary criteria to filter junior programmers add no value, but are another symptom of how broken the entire business is.
I can also tell you that most the lawyers, bankers, consultants and doctors at my university will have been doing work experience + shadowing from around 14-15 years old. The even more ambitious ones will have been doing slightly more enterprising things (entering school/local/national entrepreneurial competitions), or reading around their interests, or joining/running societies.
When looking at the most successful people I know around me, what becomes apparent is that they started way earlier than everyone else. Look at Zuckerberg, he wrote Synapse when he was 16. It's unlikely you'll hire a Zuckerberg, but when someone started coding is a pretty reliable heuristic for how dependable they'll be in the job. Think of it as hiring a senior analyst to do an junior analyst's role, if you will...
Imagine if civil engineering students were taught arithmetic for the first time at college. Firms would favor applicants with an interest from a much younger age, no?
The problem with software engineering is that understanding it well is as fundamental as understanding arithmetic. We teach arithmetic from the age of around six, yet we expect that starting to code only at college is somehow equivalent.
"ideally they'll have been programming since before college"
Lot's of people get into computer programming as a second career. Rich Hickey, who created the Clojure language, studied music when he was in college. Later, when he was working at a music studio, he got serious about writing code in C++, and from that experience he developed his opinions about the flaws of object oriented programming, and the possible benefits of immutable data.
Your remark says a lot about the hunger for stereotypes in the tech industry. As if everyone is suppose to follow the same path to a career in computer programming.
With my prior employer, I briefly had the privilege of working with one of the smartest and most driven junior devs I've ever encountered, who'd been a professional figure skater in a past life and was fresh out of a Ruby bootcamp. This was a Node shop, and I've never seen anyone else go so quickly from zero to fully productive in a new stack. If the various dysfunctions of that organization included a prejudice toward long-term hobbyists, I still wouldn't have.
I'm saying that early coding implies a higher probability of having useful experience in a junior dev (i.e. straight out of college). You seem to be suggesting that I think that the probability of someone having useful experience is lower without early coding. But I don't think that at all; I don't care whether useful experience comes before or after college.
I think the "before college" comment is mostly meant as "self taught" rather than literally "before the age of 18".
It is absolutely the case that a good developer will have perhaps a few interviews, and then pick the best offer available. Meanwhile, a marginal developer will go to interviews for months until something sticks. Thus, of the candidates I actually see, I expect the ratio to be perhaps 50:50 whereas perhaps 95% of all engineers are actually qualified for their jobs.
When a "senior software engineer" candidate can't read ten integers from a text file and print their sum after 45 minutes in front of the language of their choice, I have to assume that I'd have an equally bad experience working with that candidate in the future, and pass.
The article leans too heavily on "great developers are sticky," which may or may not be true, but the above dynamic is the main reason we have coding screens for programming jobs.
Nobody has shown me a better way of screening that actually works any better, and the cost of a mishire at the senior level is high, so on we go with the tools we have, primitive as they may be.
I don't think i could type that off the top of my head. Sure, the normal java stuff public class Foo { ... blah blah blah ...}. Integer parsing with Integer.parseInt(...); do the sum in a loop, couple minutes maybe. But i have to look up references.
File because one of the constructors overwrites whatever is there there, and the Reader family because i can never remember what i need in the stack. I don't think i've had to open a file this year.
I guess it's better to just type in everything you know, then explain what you're looking up and why. But really at my desk, I'd instantly know what i needed to look up and go look that up before typing. Rather than local javadocs, i'd google for FileReader. But in an interview i'd feel bad.
Ah yes, and after actually doing it, i'm an idiot, and there's a nice convenience method that is perfect. Files.readAllLines.
Also, figuring out the package prefixes without an IDE kind of sucks.
Google has ruined me.
But yeah. stock linux + jvm and no internet connection? maybe 15 minutes? 5 minutes to figure out where the javadocs are, another 5 to find the docs. 5 for typing. Eh, maybe another 5 for remembering the weird path conventions for javac.
I'm kind of embarrassed it would take me that long.
edit
Oh gosh. you wouldn't have ctrl mapped to caps lock. I'd look like a complete moron.
Being a weak candidate and being an inexperienced candidate are two different things.
People with average talent can become quite good, but just like everything else, they won't be able to catch the more talented person who puts in as much effort and receives as much coaching.
However, if you look at things like sports, the margins are really fine. A 2 minute margin of victory in the Tour de France represents less than 1% of the total time in the race. People at the top end of sports have the best training, the best coaching and the best talent because 1% is the difference between winning the championship game and not even making the team.
The margins in programming are huge in comparison. Someone with average talent who has good coaching and works hard in a consistent way will be so much better than your average developer that I think talent barely works into the equation. Just like everything, the super talented are rare, but in our field it doesn't matter because we can potentially train people to be much more than "good enough".
You have hit the nail on the head, though, with respect to the problem. Training is a cost centre. Let's say you take a person right out of a boot camp -- they can write code, but have only a couple months of practice under their belts. While they can improve dramatically over a year or two, it's still going to take 5 years or more for them to hit the "much more than good enough" level. If you want them to be able to handle the nuances of making sure a project is successful, you need at least another 5 years on top of that.
People change jobs every 2-3 years on average. You are just starting to see some payback on your investment and they are gone. Even worse, investing in training costs you more than buying talent. If you spend time and money (and factor in the cost of lost opportunity by hiring people who are still learning their craft), when they reach the level of a more experience/talented individual they want the same pay!
So you can pay double for someone talented who has 5-10 years of experience, or you can try to train someone up to that level. After 5 years, they still want double (if they are even still around), even though you have invested in them.
This is why the top teams are willing to spend ridiculous money (2x the median) on top people and are unwilling to train anyone. The teams who aren't willing to pay top dollar also are unwilling to pay for training. They think they can get twice as many people if they pay half as much and will get more done. They aren't interested in paying for increased skill.
Unlike sports, 1% difference in ability has virtually no meaning in programming. This means we don't need to scour the world for the best young programmers and build a world class training program for them. At best, we pay double for people who have the talent and/or experience to be "much better than average".
It's broken, but I don't know how you can fix it.
In my experience, the ones that were 'capable' from the get go either had a lot of experience programming on their own, or had some internship or other real world work experience. The ones less capable didn't.
Then I realized that my measure of 'capable' was probably unfair, because I expected people without having worked before to be aware of all the complex details that working with other developers entail. All the annoyances you have to deal with about writing real code that needs to evolve and change that only a stable work environment provides.
Then I took it one step further. What if the ones that weren't out of college were just never expected to really improve. They never had the chance to be exposes to a stable work environment that required growth. They just either were let go from previous jobs before being allowed to grow, or weren't put in an environment that neither required nor promoted growing? I think not all jobs do a good job of demanding and providing space to grow. And that's a problem. It's possible to 'get good' on your own, but it shouldn't be expected; and it's far from the only way.
So maybe we are being a bit unfair when we say a candidate is weak. Maybe they just haven't been in an environment that would require and allow them to improve.
I'm sure there's a definition of "weak candidate" that perfectly captures "uncoachable". But my definition, and I think it's the common-sense definition, is that a "weak candidate" is one who gets as much negative feedback as positive (Joel Spolsky famously advocates for "if you have to ask, the answer is NO HIRE"). Most of the people we hired at Matasano for several years fell into that category, and most of them wound up outpacing the strong candidates (none of them did badly).
For all the alleged shortages, can it really be that all these years later companies are still unwilling to gamble on training because they feel like people will leave too quickly?
It's probably not true in 80% of cases, is true in 20% of cases, and the 10% of cases where people got burned by a poor candidate who couldn't improve are the ones they remember and say "never again;" likewise the 10% of cases where they are amazed by a highly skilled candidate and think all of them should be that way. They're missing the majority effect that impacts the productivity of 80% of their workforce.
This is why it's crucial to have an innate sense of human variation and psychology, a la W. Edwards Deming. So, in that sense, it's not the employees that need the training, but the managers.
In software dev, as much as we have stack overflow and such where we can figure out trivia of the tools we have to work with, I guess we'll come to the realization that some kind of formalized training post-hire will be useful, especially as development becomes more specialized or domain specific.
To my mind, that's domain knowledge, and domain knowledge can definitely be taught, and much faster than how to write good code in good time.
I think sending some pen-testing books to a bored and under-exercised VB programmer may well get good results; but put the same guy in front of a problem that is almost completely abstract, and if he's able to understand the solution you present to him and create code (in any language he likes; we don't care about specific language experience either) that implements the solution, then he'd get hired at my company too. Because we don't test for domain knowledge; we don't expect it. We test for ability to communicate, understand and turn concrete implementation ideas into code.
I don't accept the argument that we were hiring for an easier job than "programmer"; I think we were looking for a subset of the population that ordinary software development teams look for.
The bored and under-exercised VB programmer, by the way, helped found the NCC Cryptography practice, and moved on to one of the greatest crypto software development jobs in the industry. It's not my place to say exactly which job, but if you make a mental list of the 5 coolest crypto software development jobs that might exist, it's probably one of those five.
The problem is everyone else runs hiring processes that assume the bored VB programmers are just VB programmers. They are wrong. Joel Spolsky was also wrong about this (despite the other nice things he said about VB).
I think that's actually why your experience might be less representative, but I'm open to being corrected.
Security is generally considered harder than regular development, at least to the extent that it would typically be hard to get a security job without security experience.
From what I understand, your model was to take programmers with no security experience and train them on security. The effectiveness of this doesn't surprise me at all, because it's fundamentally a different problem than taking someone who's bad at programming and teaching them to program better.
I have absolutely no problem with hiring someone who's a good VB programmer and teaching them to do web programming. What people generally dispute is whether you can hire someone who's a bad programmer in general and "train" them to be a good programmer.
While at Borland, another engineer (Allen Bauer) shared his personal theory, that some engineers were great at reading code, more were great at writing code, and not so many were great at both. Sounds like your job required people that were very good at reading code. It intuitively feels very testable.
A previous person I hired (in a previous job, for a different company) had a lot more industry experience as well as appropriate education. They were quite capable but I had a slightly bad gut feeling, ended up hiring anyway cause I didn't want to discriminate against a seemingly capable candidate. Ended up being a poor fit: argumentative and not willing to accept that they do not always get to call the shots about how stuff is done. I think this was disadvantageous to both parties.
In the future I will definitely prefer hiring someone showing a lot of potential than someone skilled who I suspect may not be a "team player" (for lack of a better term).
For a site catering mostly to software developers, this place sure is afraid of computer science sometimes. Ooh, look, a parser --- run away!.
No wonder everything in our industry is an ugly blob of JSON.
Fogbugz is (was?) writing a language for internal use only, so it's a better fit there.
jq stores all numbers as doubles, which means it can't handle 64 bit ints (or larger integer numeric types).
It's otherwise a great tool and I wouldn't point out a negative such as this, but you did explicitly mention that you know JSON won't be fed to a bad parser.
I would posit the same applies to writing your own programming language: it may be fun to do as a hobby, or when you are learning, but may not be a good idea to use it in production to rewrite your company's flagship software.
But I would hesitate somewhat to join a team which used a language developed entirely in house. As enjoyable as such work would undoubtedly be, it also reeks of overengineering, unless the problem space is so unique that no extant language can effectively address it. Experience suggests that project management software is not such a space, and the maintainability risk of a bespoke language seems therefore very hard to justify.
On the other hand, I don't pretend to be a "top dev", by anyone's definition - top quartile, maybe, on a very good day. So perhaps I'm just not favorably enough placed on the capability curve to understand why it really is the great idea that it doesn't seem to be.
I chalked it up to being a management thing. As in "Sometimes you want to let people make mistakes".
And that, by virtue of being plunked down into the interviewer's chair through whatever chain of random circumstance... you've suddenly become innately endowed with the ability to evaluate it in others.
> Do you use the best tools money can buy?
if you only use the best tools __money__ can buy, your development team is probably really bad.
You really can't get how important that list is before working for a 0/12 company. Really.
The biggest factor I didn't consider: People who exist in a 0/12 system are at best willing to tolerate it and at worst have engineered it. Change within that population will be challenging, even if they all publicly claim things need to change. You end up dancing on a razor's edge of being a positive change agent and being made a pariah. All while trying to do actual work.
Send help.
and some stuff on the list also only applies to bigger companies.
Many good tools are paid. And developers need the good tools to be productive.
Like most things, it's a spectrum.
But if you have a knack, you'll enjoy it more.
Enjoy it more, you'll do it more.
Do it more, you'll get better at it.
This feedback loop has been suggested as one root of the (contentious) "xy,000 hours equals mastery" findings.
But it's only true up to a point.
We know that in fields subject to large-grained biological differences -- sport, in particular -- genetics swamps training effects. Most of what matters is throwing people at a sport until you find the person who is freakishly good at that sport. Which is why Australia is the best at Australian Rules Football, why the USA is the best at Gridiron, why China has come to dominate the lighter weight classes in Weightlifting and so on.
I think if you look at software work you'll find the same kind of dependency on training rather than some innate gift.
They will receive positive feedback and stay with a sport longer.
OTOH I have yet to find someone who has had a Google interview that did not have major focus on algorithmy questions which you can just memorize for. To be fair, most of the folks I know are students; Google may have different tactics for more experienced candidates.
I'm sure Microsoft does this too (and I know that Microsoft is where this attitude came from in the first place), but it seems like it does it much less (from my limited anecdotal view) these days.
That aside, nothing you said actually provides any support for your criticisms. Your comment can be condensed down to the phrase "that's wrong" without losing any meaningful content.
By nature, if you post an article stating the truth of the matter - that truly good people are extremely rare and that most people tend to be mediocre and some serviceable - most people will be outraged. Because they're the "most people" being called mediocre to serviceable.
As the article points out, most jobs are mediocre. Good work is not appreciated, a company can't form a team that works normal hours and uses tests and version control, autonomy and learning is low, the work is some dull obscure business problem, IT and the company is dysfunctional and pay for the skillset is below market average. Yet all these companies want great developers as well.
Also, as he mentions in the article, sometimes a great developer does roll in but they get filtered out for some reason. In my experience it is usually due to age - a great developer who is 52 comes in, but management almost openly says they want someone in their 20s who they can push around more when they nix the person.
So the mediocrity goes both ways. When Oculus got going they seemed to have little trouble getting some of the best graphics programmers (Abrash, Carmack) to join them. If you want a great programmer you need to be a great company with a great position.
Exactly. 95+% of all developer jobs are not particularly novel nor do they require the top 1% of developer talent. However it seems like 95+% of all developer jobs believe themselves to be novel and requiring the top developer talent.
It's no more innate than any skill. The only innate aspect to it is the desire to do well at it.
> that truly good people are extremely rare and that most people tend to be mediocre and some serviceable
The problem with this sentence is the word mediocre. The vast majority of developers are average, which is to be expected. The problem is that Silicon Valley is seen as the norm for development and not the exception. SV plays right into the personality culture we seemed to have built (thanks to social media), so people start thinking all development is like that, whereas most people just want to do their job and gohome to their family at night.
... and the autism to sit at a computer all days and week-ends to practise.
You're not giving enough credit to the whoever came up with the infamous Microsoft why-are-manhole-covers-round style of interviewing.
https://www.youtube.com/watch?v=jFdJsZKyPA8
After watching it, you should be able to handle your next interview like a pro.
Sure it's near impossible to find 'great' developers, but most companies don't need great developers, they need good competent developers and there are plenty of them on the markets at any time. I would say the positions that require a 'great' developer to work on are as rare as great developers themselves.
Is this a joke? "All-day" interviews are so common as to be the usual expectation.
And while they aren't always a full 8 hours, it's not unusual for them to be 5-7+ (including giving candidates a lunch, and interviewing them during it while they eat).
They never called back.
This does not seem to vary across company sizes, locations, job level, etc. Everybody copies this horrible process.
In my opinion, engineering talent is based on three things:
- The range of projects that the engineer has worked on during his/her career.
- The total number of hours spent coding in their career.
- Their attitude towards learning.
I think this 'innate' 10x developer idea is just bullcrap.
The best way to identify a great engineer is to check how many years of experience they have, how many companies they have worked for (more variety is better) and their attitude to learning. If a developer says "I like to be challenged by my work" - That's generally a good sign that they enjoy learning; combine this with their total years and range of experiences and you can infer how much they know today.
Investment banking, management consulting, doctors, lawyers (probably) ... - seems like the traditional 'power jobs'
Source: https://www.google.com/amp/s/amp.businessinsider.com/how-goo...
The technical interviews are totally different and are used because they actually work well, but you can't do something similar with business skills.
The situation is of course different for boutique firms, I should've said that. Small firms cannot afford training candidates internally, they'll want to have technical knowledge.
I'm not following you here. I guess you are referring to a high selection bias for CS knowledge? How does that relate to Joel Spolsky?
Though I'm a much better dev now than when I started in the 90's I had my choice of jobs back then. Now it is horrible death march for the privilege of slinging HTTP APIs, about the easiest thing a developer will ever face professionally.
It's the reason why developer interviews are such a shit show, and a factor in the "engineer shortage." People who can't code with a gun to their head (ala swordfish) are excluded. Not to mention those that don't "fit", such as minorities, women, and elders.
In fact, it's hard to tell for sure how good a developer is before hiring, but easy after having worked with them for a while. This is similar to how it's hard (as in, expensive) to be sure if a car is a lemon before buying it but easy to tell after putting 10,000 miles on it. No contradiction.
Unrelatedly I agree anecdotally that from a developer point of view the market for workplaces looks more like a market for lemons (it's hard to tell if a workplace is crazy before joining it, easy afterwards, and a developer will stay a lot longer at a nice workplace than a crazy one, so most job openings are probably for places that have a high turnover for a _reason_).
> Just for example, there’s someone
> I’ve worked with, let’s call him
> Joe, who’s saved two different projects
> by doing the grunt work necessary to
> keep the project from totally imploding.
> The projects were both declared successes,
> promotions went out, they did a big PR
> blitz which involves seeding articles
> in all the usual suspects, like Wired,
> and so on and so forth. That’s worked out
> great for the people who are good at
> taking credit for things, but it hasn’t
> worked out so well for Joe.
However, I wouldn't say that there is a "market for lemons" for employees. Performance may well be difficult to evaluate during an interview process, but after several months of work within a company it is generally quite clear who is performing well.
I think the real issue is that:
- Employment opportunities are a "market for lemons", and even if a team or project is great to work within at first, it won't stay like this forever.
- If a company spots a great performer they generally don't wish to offer them preferential treatment (more autonomy, ability to learn something new, pay rises, etc) due to its perceived effects on their peers.
There are all sorts of lazy heuristics that people use, and "good people are never out of work" is one of them. Good people do sometimes go out of work, but it will generally be to "scratch an intellectual itch" or to give themselves time to be extremely picky about their next job.
I think the way forwards is to create more supportive development cultures which allow for more room at the top (with opportunities for self-direction and with pay improving alongside efficacy) and at the bottom (with mentoring and training). This isn't just something that might be solved by a great manager. It's also something that we I think we should hold ourselves accountable for.
I worry about this a lot when I'm interviewing candidates.
Basically, would I have given myself a no-hire?
Sometimes I think I would've.
Let's spice it up with 10x HR people, 10X CXO's and 10X VPs and hell while we are at it why not throw in the 10x Barista to get Starbucks HR fired up.
Here is a better idea, forget the 10x developer. Show us the 10x code.
That recognition has come after decades of exceptional work and experience, not by solving test questions in an interview process.
How are you going to find Torvalds or Carmack in an interview process before they have had the opportunity to do all the decades of work and gain the recognition?
This 10x fantasy needs to stop or let's see some 10x code.
Anyone seeking a 10x engineer needs to ask themselves whether they can afford them and if the work on offer is exceptional enough to require that level of talent.
Comparing impact of regular line developer with leader is comparing apples to oranges.
Becoming leader is a different thing - not everybody wants to or have what it takes to do it.
Hunter and Schmidt did a meta-study of 85 years of research on hiring criteria. [1] There are three attributes you need to select for to identify performing employees in intellectual fields.
- General mental ability (Are they generally smart)
Use WAIS or if there are artifacts of GMA(Complex work they've done themselves) available use them as proxies.
Using IQ is effectively illegal[2] in the US, so you'll have to find a test that acts as a good proxy.
- Work sample test. NOT HAZING! As close as possible to the actual work they'd be doing. Try to make it apples-to-apples comparison across candidates. Also, try and make accomidations for candidates not knowing your company shibboleth.
- Integrity. The first two won't matter if you hire dishonest people or politicians.
There are existing tests available for this, you can purchase for < $50 per use.
There's no need for day long (or multi-day) gladiator style gauntlets. Apply this process to EVERYONE, including that elite cool kid. You don't want to exclude part of your sample population!
[1] http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...
[2] Technically IQ tests aren't "illegal", but the bar courts have decided companies have to climb is so high it effectively means they are. I have spoken to a lawyer about this. You should speak with yours too, before you decide to try IQ testing.
And IQ also has a correlation with social status, like it or not.
> between "qualified" candidates
I'm presenting from the point of view of the hiring institution. But still... If I were a law firm, I wouldn't hire a lawyer who hadn't gone to law school. Why would we behave differently with software engineers?
The author is missing a fundamental piece here. It's NOT that the "great" developers will stay at the same job for a long time or that the company is great at recognizing them. It's that their friends, colleagues, etc will recognize them and bring them along.
When you take a new job (at any level) at a company you like doing what you like, you will want to work with great people. Some will already be there but most likely you'll have the opportunity to refer people in.. and you'll go back to the people who you already know are "great" by whatever understanding you have.
If you make a compelling offer hitting the right points - money, flexibility, opportunity, etc, etc - then you'll get the person even though they were never "on the market" in the traditional sense and almost definitely didn't have to send a resume and "apply" in the normal sense of the word.
The premise of network hiring is that when one member of a strong team leaves, they'll bring the best of the team with them. Hiring managers seem to appreciate the aspect of this that means they can bring multiple good people on at once, without appreciating the obvious implication staring them in the face: when one of those people leaves THEIR team, they're going to strip a big chunk of the team out with them.
Network hiring creates brittle cultures. It's probably the most popular "alternative" to conventional hiring pipelines, and it's just as bad!
People don't change jobs for slightly better. It's too much risk and hassle.
Funfact: We're literally in a thread talking about how terrible are job interviews.
How is it that fear of false positives keeps coming up again and again in discussions of hiring practices?
Considering at-will employment (is that the correct term?) is the default here, you'd think it wouldn't be that big of a problem to get rid of an underperforming employee. I guess it's more of a cultural/social issue but I can't pretend to fully understand it.
People are not fungible, and depending on internal politics and procedures hiring probably only happens reactively instead of proactively. If you have to fire an under-performer, you're now short-staffed even worse until you can get another job posting approved and someone hired for it.
Lost productivity from being too picky is less visible than lost productivity from telling someone to go home.
Management is often seen as a career progression from non-management. This means a lot of managers - and especially the newer and therefore lower-level ones - are still figuring out wtf they're doing and aren't (yet) very good at it.
Second, people don't want to be known as "the lemon picker" within the company.
Like I said, it's a cultural/social issue disproportionally tilting the cost effectiveness scale.
There is a philosophical view that optimizing onboarding is equally (or more) important when compared to recruiting processes. A common blocker here is that onboarding isn't done by HR.
That's not what Joel's post said. Joel said that great developers don't apply for jobs much. A great developer could have as many jobs as an unqualified one, but they rarely, if ever, need to apply to places to be offered a job there. Think of any great dev - do you think they'd send their resume someplace? No, they'd just leave, another company would get wind of it and they'd be offered a job in a heartbeat.
Bay Area or London might definitely be like this, but there are many cases where this simply does not work like that:
- a geographical area with one strong, dominating company -> low employee mobility (not much choice), hence low inter-networking
- people who move to a new country/city and know no one there (or people whose friends tend leave to other countries, hence the networks are not that useful either)
There are only a handful of developer hubs in the world, many people work in other circumstances and might not have an extensive networks, which is neglected here.
They are able to get jobs through the resume/interview avenue when they want to leave but they still go through this channel. These developers are most likely to be underpaid. Real gems for those hiring.
Lemon markets exist for one-time transactions between foreign parties. Public records of companies and potential employees change this to a degree. A big name company may be able to hire at lower cost as it is on record it is not a total disaster. A public record can help to put a floor under the salary one is being offered.
From a developer perspective one can
- market skills in a way to decrease hiring risk (affecting offered min salary)
- market skills that are truly needed (affecting considered max salary)
- research the company (allowing one to reduce demanded salary e.g. huge solid growth etc.)
As always the power in negotiation is with the party that can afford to say no.
Was the trojan horse message behind Joel's now-infamous Great Developers meme, which we are currently about midway through the second decade of. And that's really all people should have read it as -- part sincere opinion; but part pure and simple marketing shtick for Fog Creek.††
Yet it's taken on a life of it's own, as we know all too well by now -- with no endpoint in sight.
† Such brain teasers -- and the ability to solve them right there, on the spot, with strangers staring you, and your career hanging in the balance -- being, according to JS's unequivocal belief (until it finally became woefully unfashionable), if not absolutely necessary, at least fantastically useful and effective in telling Great Developers from Mediocre Ones.
†† A great, or at least an okay company, by all accounts (aside from this single, unintentedly onerous piece of marketing shtick from the early 2000s).
And more importantly for my case, does anyone know where to look for companies that are willing to work around a college student's schedule?
The cards seem to be stacked against us in this one.
There's hundreds, thousands of companies who aren't the best, and aren't looking for the best either. Work there, get your experience up, jump around every few years (so you can eventually get a feel as to WHY those companies aren't the best), and you'll be good enough to go anywhere you want.
Or you can try to play the SV game, learn the 30 sort algorithms and learn how to speak "Big SaaS Company" speak right after college and skip that.
I personally wasn't good enough for that out of college, so I did it the hard way, and now I'm where I wanted to be all along. It took 10 years, but ::shrugs::, I made it.
There's no reliable way. You just apply to some more jobs, and some of the companies will be the crazy kind, other's won't.
> And more importantly for my case, does anyone know where to look for companies that are willing to work around a college student's schedule?
Look for medium to large companies, those often have some things they want done, but aren't really urgent.
I work for an ISP and IT outsourcing company, and we have low-priority projects like updating the internally used blog to a newer wordpress version, install some plugins to get some extra functionality, write a bit of customization code and themes. These are things where we're glad when we can offload them to a part-time developer, and we don't care up if they show up in the morning or in the afternoon.
> The cards seem to be stacked against us in this one.
Don't give up. If you can get some personal connections, finding a job won't be too hard. Go to some local meetups, like your local python/perl/ruby/java/.NET/whatever monthly gathering, and see if you can get talking to some folks. Going to meetups in your free time already puts you in the upper 10% in the perception of most people :-)
All "good" tech companies. All the startups, no matter whether they are early stage or already raised 100M. All the big established tech companies (Google, Microsoft...).
To put it bluntly. ALL the cool places you know and want to work for. They are out of your league :D
If you want to start elsewhere. Look for old school NON tech companies. Things like insurance-health-government-defense. They've got real work to do, sometimes more interesting than the Google-Facebook-Mhatever. But they don't have the culture and they have easy interviews.
Remember that the reason all of these blog posts you're reading exists is because the authors are growing their referenceable public body of work to the same end, not necessarily because they're trying to help you.
Don't sell yourself short. As a fresh college graduate, you have a totally clean slate. All companies, even the big 4, will give you a fair evaluation (or, at least in theory, try to) based on what you've been devoting your full time to studying for the last four years. I assure you that it becomes much, much harder to study anything or do much alter your career trajectory once you start working full time.
Yes, you can find companies with easy interviews but, as user5994461 and others have pointed out, they tend to be somewhat mediocre, to put it charitably, in environment and pay. Do you really want to deliberately aim for mediocrity before your career has even started?
But also, try a few technical interviews. You might find they're not so bad, if you have the right background. I've been on both sides of the table in technical interviews, and on the hiring side I was working with an ex-Googler and our process was heavily influenced by Google practices. All the interviews I've done, on both sides, have been concerned with technical communication and the ability to develop a workable solution with a few hints, not with regurgitation of specific algorithms.
How do you know it isn't there other way around? That networking is crucial to building a reputation as a "good developer"?
Hence practically any engineer will be part of their organic network, whether they realise it or not.
Plus, much of that gossip is going to be with co-workers, which doesn't do you as much good when it comes to finding a job with people at another company company.
Of course it does, because people don't work at the same company forever.
By far the biggest contributors to my network are former co-workers.
You obviously haven't spent time in the defense industry.
And yet they still interview you like they're pretending to be SpaceX or NASA or something. 5+ hours of programming brainteasers (which they swear are just 'simple programming exercises' that 'are just to give insight into process' -- but are actually brainteasers, and you are actually expected to solve on site).
Which makes the inevitable "we like you, but have to pass on you for technical merit" rejection sting pretty badly -- I have the technical capability to handle these jobs, because I already do this same work every day. But I get told my technical knowledge isn't strong enough to work on their apps, because of stumbling on some of the latest trick questions.
And then, their managers write a blog post on Medium and complain about the lack of tech talent, or how they are inundated with fake applicants who "can't code FizzBuzz"
sigh
Technical hiring is just totally screwed up, and no one seems remotely interested in fixing it.
One of the mottos painted on the company wall was "We don't want people here just for the money." The CEO put it up because they couldn't pay a competitive rate. The hiring manager insisted they paid well. A bunch of devs left claiming they were paid near the lower end. For example, one dev received only a $1k pay increase when she was promoted from SDE1 to SDE2.
How can perception be so different between CEO, managers, and developers?
It's not perception, it's marketing.
The interviewer cannot tell: "We pay under market. If you go interview at almost any other company in town, you'll 20% more easily". People would just not work there.
What happens instead is, they either lie straight to their face or oversell the "culture".
In your example. The CEO going for "people don't work for the money" is classic bullshit meaning they have low wages but it's a good place (even though it is not). The manager saying they "pay well" is a straight lie.
People will believe it most of the time. They can keep employees around for a while. They do what they are supposed to do: Run the company, whatever it takes.
I would argue that a lot of (if not most) companies won't even know how to gauge long-time greatness.
I tell my students that if they want to do something where they can be gauged as individuals that they should not go into engineering unless you plan on also getting an MBA and working for a big, stable company with a long track record of nurturing talent.
no matter who you are, most of the smartest people work for someone else
Managing one team is hard. Setting up an environment that is great for a diverse set of people (whether we are measuring that by race and gender of just by people with different values and ways of thinking) is even harder. Many people in those positions don't spend anywhere near enough time trying to make sure they are building the right thing and instead copy each other, leading to what Dan describes.
None of this will change until we make breakthroughs into people management, something that, sadly, only the largest companies have resources to really look at, and the largest companies are often the most dysfunctional.
Steve Yegge I think has a similar post but from a different angle and he calls this problem the interview anti-loop. Depending on who ends up on your interview panel you might get high or low marks across the board and it won't be in any way correlated to your actual abilities.
Making good teams like Dan says is a hard problem mostly because recruiting is a hard problem and letting anyone other than the team handle the process is why things are broken. If you want to work with great people then you personally have to be actively involved in recruiting great people by going to conferences, going to meetups, writing blog posts like this one, etc. But that sounds very time consuming and most companies don't think recruiting is actually an engineer's job so they never properly budget for it in terms of training and other resources.
Neither of those is true.
Good developers leave companies but they apply selectively to companies and find jobs quickly.
Management often believes that they understand hiring, and that their mechanisms for examining performance are good. If Joel believes that one of his employees is fantastic their requests will be heard, and they will be better compensated. Dan instead is seeing developers that he knows are awesome and can't find work because Dan can't hire them.
I for one think both are right in some fashion. People that management considers high performers stay long and are very well compensated. People who are not valued that way by management might still stay (if they are not valued in the market), but will jump ship after a few months otherwise. Willingness to stay depends on getting what you want out of a job REGARDLESS OF PROGRAMMER QUALITY!
I have worked at places where people with tremendous flaws were considered high performers, making the place shed great developers while keeping bad ones. Other places just look at a specific set of things that might make someone good, and end up with unbalanced talent sets. Imagine you hand tasks to single developers, and value getting them in production as soon as possible. Stability, readability, maintainability will go down the window, and eventually said company will either fail or have to hire a different kind of dev, and force those people to win a culture battle.
So ultimately, I think this is all about how difficult it is to evaluate programmers, and how people with one evaluation function look at a world that either shares their evaluation function, or differs significantly from it.
Interviewing of late is filled with randomness. One place I interviewed, the Recruiter told me to expect coding exercise. The call starts, and I am handed out a system design problem ! Never expected a positive outcome since I hadn't prepared. Randomly, I got selected for the next round. This time around I prepared for anything come what may. The call starts, I get asked a coding question. I solve it in about 20 minutes, and the remaining 25 minutes we have a good discussion about the company dev practices. Randomly, this time I am not selected !
Another company I interviewed with last year but didn't make it but kept in touch with the recruiter for re-applying after a 12 month cool-off period. Come the time to re-apply, the good recruiter left and now the current ones keep politely refusing my application.
Another interesting one wanted me to record a video of mine explaining what I know about the company and why I would be a good fit and send that video to them ! (this one is a mixed bag, I kinda liked the idea but didn't bother eventually)
Another one had me fly out internationaly-all-expenses-paid after three Skype rounds,on the day of flying back I left a courtesy email to the recruiter thanking for the opportunity to interview and asking about possible turn-around time for a feedback. Haven't heard back !
Overall I have begun to get the feeling that at I particularly am a Lemon (or atleast the Potato no one is buying).
Was that a long comment ? Or maybe a rant.
To follow the used car analogy, good cars do occasionally need to be sold, but when they do they get taken quickly by a friend or through word of mouth recommendations. The cars that sit on the lot for the public to peruse are not great deals.
Also, Joel was a product manager, not a developer or an engineering manager.
Because they are not on the market, and they're very picky as to where and most importantly, with whom they will work. Does someone out there honestly believe that a guy like Keith Wesolowski, Dan Price, Jerry Jelinek, or Jeff Bonwick will just shop themselves on the open market? If so, it's time to do some extensive research on the subject, because nothing could be further from the truth.
Steve Jobs once put it like so: A players hire A players; B players hire C players; and C players hire D players. It doesn't take long to get to Z players. This trickle-down effect causes bozo explosions in companies.
And boy was he right; I've seen good companies and good, interesting jobs go down the toilet so fast, my head is still spinning from it all, years later.
I know how that is first hand: I don't shop myself around on the open market, people call me, and by people I mean people I have closely worked with in the past, people which I know exactly what they're capable of, what they know, what they don't know, and if I accept, the whole interview process is nothing but an empty formality. It's truly eye opening just how much of a sham-theater the entire inteview games are. The key takeaway is, if you're good, you'll have a renomee, and you're unlikely to be on the open market - you come to people and people come to you. You get the job long before any would be competitors have even had a chance to apply ("we must satisfy human resources policies"). Dan Luu's question is, by experience, an unlikely occurrence in this particular industry.
At some positions, the demand is lower than the offer
Is it just a competent developer who has effectively networked and marketed themselves?
1) How personably the candidate is. Do you want to work with this person? Does he/she have basic social skills?
2) Do they have a coding-related passion project? It almost doesn't matter what it is, as long as they have one.