The myth of the developer that can't code
neilwithdata.com
neilwithdata.com
> When an accountant is in an interview, they're not asked to multiply on-the-spot in their head what 485 x 761 equals because "you have to be good at math to be an accountant"
This is an ignorant statement. Accountats don't have to multiply things in their head day to day. Yet developers do need to code, day to day.
I'm a hiring manager and the reason people code on interviews is - well, after joining a company, the first few months they _will_ spend a good chunk of their time coding. And the companies I worked at, who dropped the coding interview always regretted it, through mishires. People who could code, but were not very good at it. E.g. their code was sloppy, incorrect and they needed considerable mentoring and coaching to write code that worked. As senior developers.
The other reason is to even the odds. 70% of people I've seen hired did not have public Github profiles or code they could show. The people who complain about "why can't they just look at my portfolio" and (potentially rightfully) conclude that they're amazing developers don't realize that this approach would result in more bias for hiring this group of people. Good luck maintaining a fair hiring bar this way or hire diverse candidates.
This is why the coding test exists in most places. It's to test for something that you're expected to do from day one and to even the bar that people need to jump through, for everyone.
I'd love to hear what OP suggests to replace the coding tests with - a chat about past projects, then make an offer? And what do you do with the people who don't have these things to show?
It does not work at all when trying to recruit a senior. I am still clueless about that. I have at least two recent cases in mind of a senior hire that lead to catastrophic results. People that new how to present well at a superficial level but did really poorly once they had non trivial stuff to do.
I mean it's not about writing "hello world". If the person is not dumb, you won't find out in an interview if someone can really code. (repos are much better for that, but can be deceiving)
"We have a 3 month probation period and if something is really not true it will have consequences. I'm crystal clear about that."
I mean we are all humans, but there is also a contract with money involved. I have to look out for the company and it has to be fair for all the other team mates/colleagues too. If you hire some "faker", without consequences it will destroy the whole team balance. (Additionally there is a "meet the team" to address the consent process). From my experience, technical people have a low tolerance for people which can only talk...(in a technical team).
So it does not mean you are safe if you can make it through the inverviews with tricks. (it may seem a bit hard, but I don't like the general acceptance of "lying" or superbeautify ^^ in the application process)
And then I talk about what we are working on and how we are structured. So the interviewee also gets an impression what the day to day work will look like.
What I really want to see, is the ability to solve problems. Because this is our real job. We are solving problems(with code). So I ask questions you can not solve in 5 minutes, but you see how someone approaches a problem. Is the person collecting intel about conditions and constraints or is he/she immediatly starting to code etc. ... For that I generally take real problems we had and we already found a solution. And sure I tell them, that it is not possible to find a solution now, because many interviewees are "conditioned" (or used to) to give you immediate solutions. Else they would panic or getting frustrated.
In general ask a question: Why should I want to barbecue(or embarass) a possible colleague?
You also have to take into account, that there are people which don't know everything yet from the specifics you need. But in the right environment with support of colleagues, they become really great. It's more about the mindset & willingness to learn and also the support of the team members.
The above is more for a general developer.
For an expert with 7y+ (whatever..) experience in a specific topic. I would really ask very specific questions and maybe a "homework". Because I hire an expert for specific experience in a topic. Also the risk of losing money & time is much higher. Because it's experts work...it's hard to judge if the person is not capable or if it really just needs some extra time.. ;-) You won't believe how many people have a lot of years of experience and made only basic stuff during that time. So on paper they should be an expert, but in reality they are not.
It isn't until I read your answer that I realized this: we were both convinced because the guy was smart, clearly, however he didn't know enough of the technology we were using to develop anything meaningful with it. While when hiring a senior developer, it's possible that they can pick-up a new language or framework very fast, this is highly unlikely with a junior (this is not something I have in numbers, just thought).
The real side of the problem is that is very hard to figure out if someone will be good with everything that's not "oh, you can code". It's not well defined in the industry, so there is no objective methodology to evaluate the skills of a person. As a developer, you can usually recognize if someone can code, but determining if they are sloppy, care for design, that's a totally different thing: you need to work with them to figure that out, this is probably why an interview is not enough.
> The author does not mention his experience interviewing or hiring people
From the previous article: "I've been a software developer now for around 15 years. In that time, I've been a day-to-day developer, run my own successful software business, was co-founder at a VC funded startup, and I've had to hire and manage developers."
> I'd love to hear what OP suggests to replace the coding tests with - a chat about past projects, then make an offer?
From the article: "Why can't you look at a developer's portfolio, education, projects they've built, GitHub repo, talk with past employers, check references...and from that make an informed hiring decision?"
I have no Github, portfolio, nor education. My previous companies have a bad engineering culture. So they’ll give me good references, since they don’t know any better.
I have trouble writing a fizz buzz and yet that system could have me hired.
I've been doing this for 25 years now and I've never had to implement an algorithm[1] from scratch or do any dynamic programming. I'm not entirely sure I'd consider myself "clueless" because of that.
[1] In the traditional computing sense of the word
This depends on what the exercise tests. For datastructures and algorithms, yes maybe.
We use a test where the most advanced data structures you will likely use are dictionaries and sets, and even those aren't necessary. The main thing the exercise tests is how do you structure a chunk of non-trivial business logic such that other engineers can understand it and reliably contribute to it.
We assess this through code review, looking at the architecture and abstractions used.
"Most recently I was the CTO of RateIt where I probably interviewed over 100 developers."
Making it easier to fire mishires would probably cover it.
Accountants don't need to show they are qualified in interview because they have the letters ACA, ACCA, CEMA or whatever that they can clearly point to.
Lawyers and Surveyors similarly have profession bodies that handle qualification.
Software engineering is not uniquely broken but it is broken, but the author jumps to the wrong conclusion.
The other engineering and professional disciplines have professional bodies who have official qualifications.
Software engineering has never solidified around any particular set of qualifications and many think they are not relevant.
And artists are often asked to do some initial work for free before a client commits, they often end up doing a lot of "wasted" work for free without getting properly compensated, so it seems odd to argue that's a model that software should look toward.
- the state of the art is both extremely diverse and rapidly moving, so any credential system would run the risk of being too specific and easily out of date. Very few are respected and most existing ones are tied to particular products. Possibly the only set of those that is respected is the Cisco ones.
- most of our work is (a) tightly intermingled with the work of others and (b) copyright belongs to our employers, so we can't build portfolios like artists. Many people have contracts that purport to own all code produced, even in your spare time.
It would be great if we could take all the basic tests once, upfront, under exam conditions, and then re-use that for years. I can't see how it can be coordinated. Maybe what we need is not a union but a professional body. In the UK we sort of have one (BCS, and more absurdly, the Worshipful Guild Of Informational Technologists) but their impact and benefit is also pretty tiny.
And that just because their stack is slower-moving. This is a problem that unfortunately is not going to be solved anytime soon, I fear. As industrial revolutions go, we are still in the "mad inventor / amateur / visionary" phase, really. It will take another 50 years before the field slows down enough to solidify into a consistent curriculum.
> we can't build portfolios like artists.
That's basically what personal Github accounts are, de-facto.
It would be very different if more employers paid more than lip service to professional development and encouraged the publication of personal githubs on "work time" as part of that. It sounds like (some of?) the big Bay Area companies do, but those are the ones after the big hiring barrier.
The actuaries I know spent a lot of time on exams, but also in their early years got some time set aside by their employers for that.
So much of your work has insane barriers to entry, even for internships, that I'm surprised your roles don't come with one or two assistants to manage the mundane business of scheduling meetings, filtering emails, and planning deadlines around out of office time.
I got my BS in Cybersecurity, and after a few interviews, I nope-d out of IT as a whole. They'd post preferred certs on the job description (like CCNA) only to ask why I wasn't a CCNP during the first or second interview. Or they'd debate whether some other candidate with a CompTIA cert was better prepared for the $15/hr. internship. And I'm nowhere near the specialization of most engineers.
Awesome, awesome work, but it takes a ton of grit to get your foot in the door, and even more grit to switch jobs.
I blame this on the tendency to reinvent the wheel instead of fixing the existing one, and the tendency to chase the last new shiny thing of the month.
On other hand, if I to have a new guy sitting next to me tomorrow which code I will review and support... Well, I'd like to ensure somehow that I won't spend hours and hours on explaining things and that I won't get paged at the midnight because of suck trivial f-up.
What's the solution though? I can see anything but just hire the person for a week, 2, may be a month. Give them a boot camp project, observe and then make a decision.
That's what happens with lots of interns anyway...
I went job hunting last year, and what I was really looking forward is technical interviews - PLEASE just challenge me, allow me to show you my work instead of my CV. I needed technical interviews to fight my impostor syndrome.
I totally agree. I hate the irony of having a highly technical job but then needing some decent writing skills to put together a CV that isn't just a list of employers, dates and acronyms.
Will GCP certifications still be useful? Should Google be the certifying body for their own technology? When a field is so dependent on proprietary tech, does it make sense to certify people as metaphorical Windows Metro experts?
In 2008, I was looking for a job after being in at one company for nine years. I happen to keep all of my job search emails in a folder by year. Looking at the job reqs for standard enterprise development in 2008, they were looking for either Java, C# or a few were still looking for C++. Java has been in demand since 2000, C# since around 2004 they both still are.
As far as databases, the big four were Sql server. Oracle, Mysql and I think Postgres (wasn’t on my radar until 2012). I’ve been going back and forth between Mysql and Sql Server for two decades.
Javascript has been in demand since 2002. Definitely by 2008 when employers were asking for “AJAX frameworks”.
As far as your examples, it’s not hard to guess that you should stay away from basing your career on anything built on top of Google’s infrastructure (GCP) when they are in third place behind AWS and Azure, and aren’t really making that many inroads into large corporations. As far as Metro, by the time Metro came out, it had been clear for years to avoid desktop development of any kind outside of games if you wanted a technology with broad appeal.
I did have a "portfolio" of projects on Github, but it wasn't very large, and the most interesting project there were, well, too complicated for an interviewer to understand in short amount of time. And what about people who don't contribute to open source, either due to lack of time or desire? What do they have for a portfolio?
Don't get me wrong, technical interviews certainly aren't perfect, and there are a lot of ways to improve them. But I think they should be improved by making them measure what an developer actually does, rather than removing them altogether. In fact, I would say that if accountants (or any other technical profession) aren't assessed on their technical ability as part of the interview process, maybe they should be.
While I was there they shifted their hiring practices, focusing instead of merit and the results of a technical assignment and interview. We ended up hiring early 20-some year olds fresh out of college because they wrote great code and could explain it to us. I mean we've had some people that made a solution that sorta worked but couldn't really explain some of the work they did, or the underlying logic behind it. Some had great CVs (I mean that's the first check), but there wasn't any skill behind it.
She has had one person last more than three years on her team. But with 1/6th of the workforce of any other manager, her people have ranked top three annual billing and efficiency for the last 6 years.
There are tests, but they're unconventional. And there are times I hear my mom pick up her office phone, call one of the firm's partners and say, "Stupid question. Would you allocate this to X or Y?" And then 40 minutes of heated debate over whether it will or won't cause a $300 penalty on a multi-million-dollar return.
Maybe engineers need more unconventional tests?
I suspect that's wrong. To get a job practicing law, someone generally needs to have passed their state's bar exam. Similarly for other licensed professions like doctors, CPAs, civil engineers, etc. But anyone can legally call themselves a software developer.
IT is the far west. There are few educational requirements, and hiring laws specific to our craft. It's a very young field, unlike medicine or architecture, and we just beginning to establish professional standards on the job.
The fact most people see it as magic and can't evaluate it at all doesn't help.
Anecdotally, I've seen many programmers that can't code.
Remember that 2 dev projects out of 3 fail, for a clusters of reasons. And when it doesn't, there is sometimes a few hard kernel of devs that are doing most of the important part of the job, fixing other's mess on the fly, or just plainly doing it for them.
So those bad programmers can go from failure to failure, and never be blamed for it.
But honestly, it it weren't that way, I couldn't have entered the field. I've been given my chance more than once, and I've screwed up many times, before even becoming a decent professional.
Let's say I give candidates a basic whiteboard problem that requires no fancy algos/data structures other than knowing about arrays and dictionaries. A passing candidate will demonstrate the following attributes:
1. Understands basic programming constructs and elementary data structures (loops, vars, arrays, dictionaries)
2. Can model problems in her head
3. Can communicate effectively
4. Can work well under the pressure of an interview environment
So it's process that selects candidates that have desired attributes 1+2+3 but the downside is that it also selects irrelevant attribute 4
This is a pretty good system! Sometimes it will fail because candidates will be rejected for lacking 4 even though I don't care about it. Is there a better system? I don't believe that just looking at resumes and past experience selects for 1+2+3 nearly as well.
Furthermore I've said that "works well under the pressure of an interview" is an irrelevant attribute, but I think you could also make a case that it is weakly relevant in this way: if a candidate can do 1+2+3 while under pressure then you would expect them to perform 1+2+3 as well _or better_ when not under pressure.
There is also desired attribute 5 "is pleasant to work with" — coding interviews don't really select for this, but I don't think anything else does either short of probationary periods.
A take-home test would cover 1+2+3 without 4. With the caveat that 3 becomes a test about written communication, which may be a good or bad thing. If you want to do both, you can expect a written description of the solution, but also go over things verbally if they pass the bar, in next phase.
Recently I went through the process of an internal transfer in my company.
I applied to a team and went through a loop, just like an external candidate would.
I got rejected by that team. The reason is that I failed their systems design round.
The funny thing is that I spent the last two years maintaining and optimising the backend of my current team, that supports hundreds of thousands of customers.
Let me be clear, I didn't do well in the interview. They asked me to design a Dropbox clone and I flunked a few points. I didn't know long polling, and I tried to implement some of the features using the same solutions that we use in our system, which don't work very well here.
But what is the point of the interview? Is it to assess that I can design a Dropbox clone in 50 minutes, or that I know about systems design? If it is the later, you can just take a look a my work. You have access to all my pull requests and code reviews, design documents, and peer feedback. Heck, you can even ask me to walk you through our system and explain how and why we implemented certain features.
The reason why internal transfers are set up like that is so that internal candidates don't get a better treatment than external ones. Sure, I agree that some checks are in order to avoid moving low performers around, but are we trying to hire talent or make people jump hoops to get a job? The best part is that the hiring manager recommended me to 'reapply in a few months' to the job. What does that mean other than 'go study a bit and come back'?. If that isn't broken, I'm not sure what is.
I get that people aren't good at coding in interviews (count me in!), but I have lost count of the number of applicants who can't get it done. The situation got so bad that they kindly asked me to stop asking such questions.
I used to have a set of unit tests and asked the candidate to make them pass and I thought this was a good system.
We have a senior dev that got something wrong early on, started getting more nervous, refused help as it was a simple problem, got more nervous and flaked out completely.
On the other hand we had people that were so unfamiliar with coding that this was a good test to weed them out.
One example was an external contractor who, according to their CV, had multiple years experience with programming in C.
The person was tasked to fix some errors in our code base that the static code analysis tool found, e.g. remove implicit type conversion, etc. They submitted their changes for review, claiming the analysis tool reports 0 issues now. Their code simply did not compile (and the analysis tool only works with a successful compilation)!
Even after explaining the issue, the developer asked if the task is completed now and if they could move on to the next one.
This wasn't the only time this developer proved that they "can't code".
The issue is that it is very easy to claim to have experience. There are a lot of self-taught programmers out there who learned coding at home by themselves. How do you differentiate those from people who "can't code", but claim to do?
I like the approach to let them implement a small(!) piece of code ahead of time and in the actual interview let them explain their code, their decisions and ask questions.
Once you submitted the exercise and went to on-site interview you sat with another engineer and they asked you to make some basic changes there and then.
I liked this approach: you could take your time working at home to be sure you understand the assignment and have a chance to think through your implementation, then when it came to the actual on-site interview you were already familiar with the code you'd already written so thwre was less of a deer-in-headlights effect. Bonus is you had to "work" with someone too so they can see if you are a good culture fit/not a jerk as well
All that to say if I both have a job and have two or three companies vying for me at about the same salary - I’m no special snowflake. I’ve just done a lot of targeted resume driven development and have built a strong network - why would I do a take home test?
I’ve scheduled six phone screens with different companies in one day.
My record for looking for a job, to having an offer is four days. I called an external recruiter on Monday, had a phone screen on Wednesday, in person Thursday morning and offer that evening. It was for a job at a recently acquired subsidiary of what was then a Fortune 10 company.
I'm not saying you have to be able to pass a FAANG technical screen or you are crap, I'm just saying there are a lot of pretenders that still apply for and manage to get jobs writing software.
For instance explain the concept of MVC. I have interviewed candidates that only could explain where they would put certain code based on the framework they used but could not explain the design pattern nor why they would put the code where they said the would. these types of questions will give you some insight if someone is able to reason about code and explain why they prefer some style over the other.
I work for large corporations, mainly banks.
In this setting there are couple of ways for people with absolutely no ability to code to get through the cracks. For example, one way this happens is through an external contractor vendor.
These vendors are only interested in fulfilling contract requirements and typically will try to sneak developers with least ability they can get away with. As hiring process for vendor-supplied contractors is relaxed it is not at all difficult and will only depend on management of the team/project to prevent. Then, if it is no longer sustainable to keep particular developer on the project the vendor will not let go of the developer, it is more likely he/she will be rotated to another project.
I have worked for a project in Credit Suisse where the "principal developer" was a person rotated from another project where she served as "development manager" and had many years of "experience" as Java developer before that. This person had only barest ability to code, she struggled to write a simple loop. When asked to add Json serializer to an application she was confused and wrote a bunch of code that concatenated strings to produce something not exactly resembling Json. She did not even have knowledge to spot the problems and would blame tools and other team members for parsing errors produced from parsing the code. Surprisingly, these explanations were accepted by the management as she had more "experience" than other team members.
When I joined the team and spotted this situation I implored management to reevalute her as principal developer. The management had no development experience either and were taking her explanations on face value. They told me that I need to spend more time on the project to "learn the way they are doing things". Only after months of persuasion and showing countless examples of glaring incompetence I was able to succeed to resolve this problem.
I find it not difficult to imagine more possible settings where people with no competence in coding can flourish when they can initially build a strong position for themselves and can exploit incompetence of their managers and peers.
This is especially true for large corporations which have huge appetite to hire developers and huge budgets but also can have managers with no incentive to do their hiring very well.
In one project I worked for the manager kept overseas staff even though it was net negative productivity because "higher management said so".
This was a developer who can't code. You can't tell me this guy is a programmer who's just been off "not coding" for a few months.
The author needs to take off the "happy town" rose-tinted sunglasses.
Like any profession, software development requires competence. Empathy and understanding are not a driving factor.
Being able to produce a maintainable solution for a business problem and being able to communicate that solution to others is what is important.
If what he says is true, then why aren't Google et al. hiring mediocre developers?
Take home point would be this; you are responsible for your skills. If you are in a role where you are not doing coding day to day, then you need to evaluate that role and ask yourself, is code still right for me?
Quote: "Because make no mistake, that technical interview - that one-shot coding test we do. That's it. If in the heat of the moment you make a mistake or don't pass, absolutely nothing else matters. FAANG (and sadly everyone else who has copied their half-assed hiring practices) will not hire you. Nothing overrides it: you could be a Nobel laureate and it wouldn't make one iota of difference."
> Being able to produce a maintainable solution for a business problem and being able to communicate that solution to others is what is important. > > If what he says is true, then why aren't Google et al. hiring mediocre developers?
FAANG interview questions do not constitute a "maintainable solution for a business problem" for most businesses. Unlike the author, I'm more willing to give FAANG the benefit of the doubt in their hiring practices but even then I'd give a very generous guess that only something like 20% of FAANG employees actually use the same skills in the interview in their everyday work.
Sure, Google might be able to save money if an engineer can come up with a way to reverse-index a volume of Britannica with a memory constraint of 4MB but other businesses won't if only for the fact that they don't have the organizational infrastructure (QA pipeline, canary version rollout, etc.) to properly verify the correctness of such hackery nor sufficient business clout to cushion possible outages that come with the territory of building your own libraries. For most businesses, hiring an engineer experienced with Elasticsearch would provide them with the business value they are looking for. Unfortunately, said engineer who spent the last 4 months debugging their ES cluster's split brain problems did his whiteboard interview in a Python-like pseudocode and couldn't handle memory swapping properly. So it was an unanimous no.
> If you are in a role where you are not doing coding day to day, then you need to evaluate that role and ask yourself, is code still right for me?
I used to be in this "code everyday" mentality but I realized that a better mindset is to "solve problems/puzzles everyday". Someone who works on yet-another-CRUD-app on a daily basis is definitely coding everyday but is this activity leading to growth? On the other end of the spectrum we have someone who solves ICPC World Finals problems in that 20 minutes they have waiting for breakfast to be served; honestly more impressive than the CRUD guy hands down but consider that:
- beyond something linear like lists, stacks, queues, there is very little opportunity to build a data structure class on your own. I've found my data structures knowledge useful in _reasoning about_ (not implementing) data structures other people wrote: the DOM is a tree, DB tables are trees, regexes are graphs, indices are look-up tables and/or hashes.
- I've had to work with DP in my day job once and only once so far (out of 8 years working) and that was a very special circumstance at that. And, guess what, DP didn't really save the business in that situation. Communicating and cooperating with the non-engineering departments did.
- Not to mention that sometimes life is just yak-shaving, as the article points out. I've made PRs with nothing but logs only so that three days later I can PR a one-line change (four, thanks to the linter) that would fix a performance problem I tracked down for three weeks. Or soldier through Jenkins and Maven so you can finally migrate your outdated CI pipeline.
I am now in a position where I lead an organization that hires hundreds of software engineers every year. We start our interview process with take at home coding tests as a first line screening. These tests are not hard (I am no longer a day to day developer, they are not hard even for me) and of course you can always go to Stack Overflow. ~20% of our candidates fail them routinely. I am glad we are not wasting time interviewing them.
Those are facts. There are no facts in the article. Until this industry deals with its fraud problem and certifies engineers like other fields, technical testing is here to stay.
Just out of curiosity: were these junior devs? Were they maybe more proficient in other language/IDE? Couldn't better mentoring/on-job training help - rather than just to "weed them out"?
I might be naive but (and I have also been on leading positions) I still believe that if it seems that somebody "cannot code" it (most of the time) just means that somebody a bit more senior just have to invest a bit of time to coach them...
Inevitable as they gain experience at the other company, their market value will go up quicker than their HR policy will allow for raises (salary compression) and you can still offer them slightly below market rates and they will be glad to take it because they don’t know their worth and it’s a big jump for them. They probably don’t know how to negotiate either.
I’m not making a value judgement. This is just the world we live in.
And still could not produce in real world situation, because he cant do anything except self-contained examples and gets demotivated with pretty much any task quickly. He is good at remembering what blogs and forums said and repeating it to other people.
I'm going to use this from now on to explain whiteboard interviews
How do you assess that it's their own work?
> Incompetent programmers don't have anything to show, and our industry makes it easy to come up with excuses about why, excuses that wouldn't play for a digital artist.
Such as?
One thing to bear in mind is that ultimately there is no perfect way to hire, whether programmers, artists etc ..
In other situations, the interview may be as simple as you showing up with the right credentials. Those jobs might be scarce in silicon valley or not pay as well, but they exist across the country.
Furthermore, if the procedure is bullshit, does that not mean people who are otherwise geniuses are getting passed up? If so, then there's a pool of geniuses out there to hire at a bargain, giving you a competitive advantage, should you be able to actually manage them.
Conversely, there must be companies already exploiting this advantage, so if you're a genius failing on the whiteboard, perhaps you're applying to all the wrong companies.