The Non-Programming Programmer
codinghorror.com
codinghorror.com
So I took the test and got 20 correct (as I imagine many people here would do as well). I thought it was very easy. The headhunter later told me that in 9 months, she had sent 52 people to the test, only 2 of us got 19, and I was the only one who got 20.
I'm not really sure what this means. That there are a lot of posers out there? That she wasn't very good at screening talent? That the companies seeking the best talent gladly pay $100 50 times to save their time?
I guess my biggest feeling is one of disappointment. It's just not that hard to become a good enough programmer to get 20 right every time. All it takes is study, passion, a lot of dedication, and a lot of hard work. I wish more people would do that. There aren't enough of us.
The first question was something like, "These are the definitions of variables: A string is... An integer is... A date is..." Then the question would be something like, "What can '15232' be? a. an integer b. a string c. a date d. any of the above e. a and c".
Another question might be "Here are the rules of precedence... What is (1 + 2) * (3 + 4) / 3 a. 4 b. 2 c. 7 d. 12 e. Can't tell for sure."
It got more and more complex, all in their own pseudocode. They covered branching, iteration, arithmetic operations, Boolean algebra, etc. Things most people here have seen a million times. Kinda like SATs for programmers.
My boss mentioned some time after I was hired from an ad in the paper that I was the only one who passed his programming test. Which was sufficiently generic and easy that I couldn't rememer it in detail, e.g. it was something like reverse a doubly linked list and find errors in some C++ code on the white board, none of them hard.
There are indeed a lot of posers out there.
As someone working for a big company with a lot of passion, the four items that you listed is hard. Dedication and hard work requires significant amount of time commitment. My work for 8 hours typically involves so much paper work, process and discussion that only 20% of that time is really spent in programming. Between family & kids and a 8 hour job, I try to spend 1 hour each day doing new stuff, but with that time, it takes a long time to be really good.
http://joelonsoftware.com/articles/fog0000000050.html
and
http://joelonsoftware.com/articles/FindingGreatDevelopers.ht...
Let P(A) be the probability that someone is a pretty good programmer. (I said top ten percent, so .1).
P(A') is 1-P(A) = .9
Let P(E) be the probability someone is employed. I guessed and put that at 1-P(E'), guessing programmer unemployment rate at 6%. P(E) = .94 P(E') = .06
Here is a more wild assumption. Let's say that given that someone is a top 10% programmer, there is a 98% probability that he or she is employed.
P(E | A) = .98, and conversely, a .02 probability they aren not.
With these assumptions in place, I ran the calculations. I was intending to type them up, but it would be difficult with just a text area.
Here is the punch line (with those symbols, and assuming I did this correctly, which I did it on the bus on the way home, so probably not)...
Given that someone is unemployed, there is a .033 probability that they are in the top 10% and a .96 probability that they are somewhere among the rest. (The 'rest' includes a lot of pretty decent programmers though!)
In other words, 3.3% of applicants would be top 10% candidates, and that's assuming equal probability of them sending in an application in the first place.
I might rework it with different assumptions. Anyway, you said the math was pretty simple. If someone wants to see how I screwed it up I can type it up into latex and upload it somewhere.
Ehh... I'm pretty glad we're in short supply. Makes it very easy to just walk into (or out of) a job even in a poor economy and raises my wages to levels I feel guilty for charging :-|
Incidentally, if you think Java programmers who can't figure out how to count the number of occurrences of the letter "a" in the string "Java java java" are bad, you should see how many interpretations of "advanced oral and written Japanese proficiency" we've had. Oh boy. Let's hypothetically say you're a recruiting company and you get a candidate as far as our door without disclosing that their exposure to Japanese is, I kid you not, "I loved watching Kenshin in high school" -- we will not be doing business with you again. Hopefully ever.
For me, a coding exercise is more than just a way for me to see if the person can produce something that works. It's also a kick-off point for other questions. Often there are things the person will do along the way to a solution that reveals some lack of knowledge, so I keep my eyes peeled as they talk through their solution for these potential gaps.
One of the biggest problems now is that you have people who can even pull of FizzBuzz but don't know much of anything beyond how to get something like that working. They don't understand data structures, algorithms, or how a computer really works.
A simple coding exercise like FizzBuzz can reveal these gaps too, even if they get the problem right. The trick is to use their implementation as a way to probe their thinking.
For example, if they are just using a "sort" function, like in Ruby, ask them how it contributes to the runtime of their algorithm. Or, if they are loading a file into RAM instead of just iterating over reads on a file handle, ask them about it. Or, if they are iterating over the keys of a hash (h.keys), ask them how they think that works. (This recently revealed to me the candidate did not understand that a hash function was irreversible.)
None of the major gaps in knowledge I've found that separate mediocre from good candidates were found by just looking at their code and giving them a thumbs up or thumbs down. That's good for separating the mediocre from the morons. To find out if they're good, you have to probe.
To find out if they're great, well, that's another story entirely that is far afield from FizzBuzz :)
That's a lot, if you ask me - the morons usually make up the vast majority of candidates, so by filtering them out effortlessly at an early stage, you get to spend your time more effectively - identifying the great ones among those who at least hold a promise.
I've seen a few "will pick up [perl | c++ | java | whatever] as a I go" type guys. (Come to think of it... I've been that guy on occasion)
We asked the prospective employees if the knew how to program for Windows, VB, etc...
They almost all answered 'yes', when we asked them to write a sample program the couldn't, so we asked them to describe how the would program something for windows...
Their answer: 'You put the CD in the drive and run the setup.exe'
They all thought that programming meant 'installing' software... Needless to say we ended up not hiring anyone.
I always feel like a fake - not about programming in particular, about whatever I'm doing and supposed to know about. I like to tell myself that I'm being 'Socratic' and worrying about what I know I don't know.
1. I'd send you one of my regular tests
2. You could reply with your code
3. I could tell you if I think you can code at this point in time, under the conditions you used.
Or you could email me. 30 seconds of searching didn't find contact details on the blog in your profile.
EDIT: amended 3 in light of insightful comment(s) below.
"Can't code now" could become "Is a great coder" later.
Everyone is at their own place on their path. We've all traveled that path. I look at some of my code as recently as a year or two ago and I'm embarrassed.
Nobody here should get discouraged. You should just get better.
Wrong question.
Right question: "Where do I stand and what to I have to do to get better?"
Clue: The answer has nothing to do with comparing yourself to anyone else and everything to do with what your building vs. what you need to be building.
How you describe yourself is one aspect of Dunning-Kruger: highly competent people often do not think that what they do is anything special, even though in reality, their skills are very rare and highly sought-after.
ASmartBear covers this topic from a related angle and it's well worth a read. Esp if, like me, you frequently have to battle the "I'm not good enough" demon.
I've done several interviews where I was asked to code and I really don't think it's easy, because you're under intense pressure of doing your task right the first time and under a very tight time frame. Sometimes even with your interviewer sitting next to you.
My programmers wrote FizzBuzz in under 60 seconds. If you can't write FizzBuzz in 15 minutes in an interview then you are either unable to program, or any level of stress will prevent you from doing your job. Either way, you couldn't work for me. I don't routinely stress my programmers, but stress is a matter of observable fact.
It's hard to make analogies to other fields, but it feels sort of like a writing job not asking you to write an insightful philosophy essay on the spot (legitimately difficult), but just asking you to write a paragraph of grammatical English (anyone even remotely fit for the job should be able to do it).
PS: Try composing your response by saying one letter out loud at a time without typing it before you type it.
writing a program by hand is not cruel and unusual, it's quite realistic. it's like a musician site reading, not an author writing an essay by spelling letters. (site reading is a skill that comes with being a musician. i can site read music that i would have had to learn to play in the past. i always prefer to read challenging music, but i can play comparatively easy music with little problem.)
I presume you meant sight reading. Threw me for a couple of seconds.
You don't ask an author to to write a perfect essay off the cuff. But certainly any author worth having could generate a brief outline for a story (or whatever type of writing you want them to do)
I agree completely that writing code on a whiteboard is utterly alien to some people, and is so completely divorced from their usual way of working that it's unlikely to give a true picture of how they will code in the real job.
But we're not asking for an insight into how they will code in the real job. We're asking if they can write something like, say, FizzBuzz. More, we're not even asking them to be syntactically flawless and optimally efficient. We're asking if there's any chance that they'll produce working code at all.
I agree that there is to some extent just a disagreement about where to draw the line, and how detailed/accurate the solution has to be, but my experience is broadly in line with the worst claims - 95% of people who approach me for a job in programming can't write code at all.
For the remainder I'll ask how they usually write code, and then collorate with them on something more complex to see how they work.
In practice, few people are asking questions that are as easy as FizzBuzz. I've been through a lot of programming interviews, and I have yet to encounter a problem of that simplicity as an interview exercise. Instead, most companies seem far more interested in laying down the gauntlet, so that they can convince everyone (including themselves) that they only hire "the best".
It sounds like you're trying to keep your programming interviews sane, and that's great. But the problems happen when the other 98% of programmers convince themselves that whiteboarding exercises can be used to successfully discriminate the best of the best from the merely good. One needs only look at the other comments on this page to see that there's a huge and disjoint set of skills that define "competency" to different people.
I would think an interviewer should accept someone saying "I'm having trouble on the board, but what I want to do is loop through the numbers, and for each number, mod it with 15, 3 and 5 and then print FizzBizz, Fizz or Bizz." Or, have them draw a diagram, whatever.
Of course, this would need to be supplemented with a coding sample or anything that shows that the person can code when they are in a comfortable environment.
But your code doesn't have to be correct. Not as you first write it down, anyway.
The advantage of writing the stupid and broken version of FizzBuzz first is that it gives you ample scope to demonstrate your ability to critique your own bad ideas out loud, write tests and conduct them out loud, eschew premature optimization, analyze the performance of something that's way too slow, and invent alternative approaches only after you've produced the stupidest thing that can possibly work. Not to mention your ability to stand in front of a whiteboard and talk while staying on task and not wasting time, which is a core skill in engineering, even if what you're doing on that whiteboard in real life is rarely actually writing code.
I've tried to do thing mostly verbally, testing their underlying understanding of CS concepts, and hoping this is a decent proxy for development ability.
If you design the questions carefully, the way the applicant answers tells as much as the fact that it's right or wrong.
For example, in a C tech interview I'd ask what you need to if you want to pass variables into a function, and have the function change those values for you. There are a range of correct answers from "put an ampersand in front of the variable" to "call by reference rather than call by value", and these shows different levels of understanding.
I also liked to ask about what the difference between single quotes and double quotes were. A simple correct answer is "one's a character, the other is a string", but somebody who really gets C would tell me that one will lead to the generation of an integer value, while the other results in a pointer.
So it's not just right/wrong, it's shades in between. With some careful design, you can ask questions that tell a lot about how well somebody understand programming.
But occasionally I'd bump into somebody who just couldn't get any of my questions at all, and I'd feel really bad as I tried to think of something simple enough that they wouldn't feel completely awful.
They're the ones who get the most realistic view of me as a programmer (for better or worse).
The realistic view of your ability comes later in a second, or even third interview. At that stage we will also talk about system/program/algorithm design skills, ability to work with others, willingness to travel, preferred working hours, and a whole bunch of other stuff to get an idea of how you add value to the company, and how well you will fit with the existing staff and ethos.
But if you can't write something as simple as FizzBuzz in under 15 minutes in a slightly strange environment, you probably won't be able to program well enough here in this company to bother with the other questions. Working here requires more than just producing code in your ideal environment without working with others. I suspect we're not alone in that.
EDIT: I've corrected a few typos. To the person that down-voted me - thanks - you made me read it again and realise that it wasn't as clear as it might've been. I was on my way to a meeting and wrote it in a hurry. Although still not perfect, I think it's better now.
To others who might choose to down-vote it, I'd appreciate any insights as to why. Sometimes I hold unusual, eccentric, or unpopular opinions, and I'm willing to bear the consequences of that, but I'd really appreciate the opportunity to learn what you disagree with. Please, share your thoughts, don't just down-vote without giving people a chance to consider what you think.
And if a potential candidate can't be bothered to do that, how bothered will they be to make small, uninteresting changes to working code? How interested will they be to fix a corner case they don't care about? How careful will they be about details when releasing code?
Not being willing to "jump through hoops" is an indication of fit, as much as many other things. My job is to make the hoops as big as possible, and as easy to jump through as I can make them. If people can't be bothered, then I guess I won't employ them.
Yes, it biases the selection process, but it's a huge save not interviewing people who can't, or won't, produce FizzBuzz (or similar).
I asked about the data structure several times as I thought they were looking for a generalizable solution that could be used with arbitrary objects and wanted to know if I had to define a separate class to implement a linked list or if I could just use a vector. This made the interviewer increasingly flustered because from his perspective the data storage architecture didn't matter and the answer was simple. I later found out he simply wanted to track the largest item on insert with a separate int and return that on max() - this was their concept of a good programmer, despite the fact it would be unnecessary and inefficient in any ordered list).
After a few minutes he concluded I couldn't program and moved to end the interview. I decided that if communication problems were this bad (probably my fault as well), this wouldn't be a good place to work. And I was also sort of shocked he hadn't bothered to vet any of the public coding work I'd done. Who brings people into interviews totally blind?
So while I might have been sympathetic to this article before, I wouldn't be so quick to blame the programmers. I was irritated this company had wasted my time, and probably vice versa. I don't know what the solution is for employers, but the solution for job-seekers is to require people to look at code you've produced before you meet them in person, and bring a laptop so you don't need to stare dumbly at paper while you try to remember the exact way to initialize a vector of pointers in C++. Either that or do a phone interview first to get this stuff out of the way.
The next time, I started with fizzbuzz and made life a lot better ;)
The best "interviews" aren't really interviews, though, they're recruiting sessions. A really strong, experienced software development manager who plans to employ programmers should have 1) a strong background in programming, and 2) a strong network of programmers who would jump at the chance to work with him again.
You already know your potential hires are good programmers, because you've worked with them before - in school, on side projects, at work, in various collaborations. You don't need to ask them to print all the nodes in a binary tree, and if you did, well, you'd probably lose the candidate.
I think Joel (on software) put it this way (I'm paraphrasing): if you're a small independent film maker, and Uma Thurman expresses an interest in your movie, you don't ask her to audition, you ask what you can do to get her on board. I think 37Signals is another example of a company that mainly recruits people they already know through programming projects (esp contributions to rails).
I won't say the technical interview is evidence of a "broken" recruiting system the way some people do, because let's face it, we don't all have the recruiting mystique to wave the magic wand and get top people to join us, and our networks just aren't big enough. So we have to interview relative unknowns.
But I will readily admit that this shows that I'm not doing it the best way. This goes both ways. If I'm in an interview and people start asking me how to implement a singleton, I start to wonder if I'm going about my job search the wrong way. If I have to ask the candidate this, then maybe I'm going about the recruitment in the wrong way as well.
But if that's your reality, then yeah, make sure you screen well.
But I'm not really a fundamentalist about the code test. I've gone through many, many, many of them. I just had a little epiphany the last time I was in one. I was thinking, "why don't these guys already know that I'm a good programmer? Why don't they already know good programmers?"
It made me realize as a programmer and as a manager that I wasn't really building my career the right way. I decided that I wanted to build a network and a reputation that went both ways - essentially, I wanted to have a large number of people out there, in different organizations, who are both confident in my tech skills and my value as a manager. I'll step outside that network constantly, and when I do, I'll certainly validate programming ability (and I'd expect people to do the same for me).
But every time I do this, it's a reminder that I failed on Plan A, which is the best way to do things.
How did you accomplish this?
So far, it hasn't been much more than doing a decent job in the different positions I've held, and working in a few different industry sectors. I've worked for a large silicon valley company, a couple of public sector organizations, and a couple of small startups, and I've taken on management and dev roles. I keep up my contacts.
When the startup I was working for folded, I got a bunch of interviews (lots of tech questions), but my best offer came from a former coworker who didn't bother with a tech test because he'd already worked with me.
While I wish I could say I'd done more (presented at conferences, made big contributions to open source, etc...), but in some ways this is a better story, because people start to think that if they need to be a total rock star. You really don't have to, just keep up your contacts and make sure you have a reputation for good work.
I would also like to point out that if you are up front about your lack of skills and applying for an entry level post, it can work out. I had one interviewee who had no programming skills, but was ready and willing to learn. She was up front about it, convinced the whole team of her sincerity, and willing to start for less than most people with a CS degree. By the end of her 3 month probationary period, she was competent. Still junior and with a lot to learn of course, but quite competent and a good hire overall.
Note: The author of the 'non-programming programmer post' saw took my comment down when I had posted it there (which is why I posted it here). Guess there's no knocking of the fizzbuzz test allowed... :p
If you don't know what the modulo operator is, door is right this way, operate it by placing your hand around the knob, turning clockwise, and pushing forward.
Unfortunately, it doesn't necessarily end upon hiring either. Usually, the "implementationists" won't get the "abstractionist" point of view and will reject it immediately. And since they're more common, this can lead to bad ideas winning simply because everyone understands them.
I became convinced recently with a new approach to this problem. If you want an artist, pick someone with decent interest to learn art and put a brush in his hand and he becomes a painter. It takes time but this is overall good for the programing community.
In an interview at another company there might be a complication like "By the way, A and B are Google-sized" or "By the way, A and B are being written to." or "By the way, you have to do it in constant space.", but none of those things apply to our business.
python -c "print '\n'.join(open('file.txt').readlines()[::-1])"
python -c "print '\n'.join(reversed(list(open('test.py'))))"
python -c "print ''.join(reversed(list(open('test.py'))))"
perl -e 'print reverse <>' <file>
I've been programming over 10 years and my opinion is that some of these questions aren't relevant for many development jobs.
Secondly, this question didn't ask not to use the "reverse" features in the language. Indeed, the suggested solutions do use the reverse features.
Thirdly, perhaps this particular task is a good one, perhaps it isn't. Even so, it seems good enough to cull those who can't program at all. The point with these exercises isn't to get it right first time, it's to see if the candidate has a clue at all. If they get it right first time then you have a prospect. Treasure them and move to discuss design, style, ideas, and interesting stuff.
Finally, do you know any actively employed, productive developer who cannot write FizzBuzz in less than 10 minutes?
But I can tell you I have had many clients that have been happy with my work.
<code> for (e in 1..100) { if ((e % 3 == 0) && (e % 5 == 0)) { println "FizzBuzz" } else if (e % 3 == 0) { println "Fizz"} else if (e % 5 == 0) { println "Buzz"} else println e } </code>
That's good enough to avoid having your resume deep-sixed. The next stage would depend on the position you're applying for, and whether it's algorithms, web, systems, maintenance, whatever. There are now several questions I could pursue, but that's enough to get you through the door, offered a beverage of your choice, and effectively courted.
By the way, not knowing what the modulus operator is no bar - all you have to do is write:
if is_divisible_by(n,d) ...
But you are already way ahead of the game. FWIW, when we have been hiring (which we aren't now, by the way) the person who wrote something like that got the job. And yes, out of several tens of applications, only one could do it.
My C++ and Java are very rusty, and I don't think I could do it on the spot if I tried; let alone go into languages like python and ruby that are new to me. But if you gave me a week or two, I know I could get up to speed.
I've had some interviews where they asked tough questions and I got a job offer but I declined because I felt there was no rapport or interest in my career.
It would probably be easier to lie your way (with mere knowledge and possibly some experience) into the position of a lawyer, an airline pilot, or a medical doctor than a programmer.
(At least I hope they don't test doctor applicants as like "Please operate this unfortunate cardiac surgery patient as an example so that we know you'll know how to do it.")
You should be looking for competence by looking at their programming accomplishments, both in and out of work, as well as their ability and desire to teach themselves (probably the most important trait IMO).
How about asking someone how they got interested in programming, or what sort of things they've created with their knowledge in and out of work, or what languages/technologies they've taught themselves in their free time, how they go about fixing puzzling software problems, or what they consider to be good vs. bad programming practices, or even what their hobbies are? - those kinds of things that will tell you IMO whether they are smart, passionate, and will make a good employee.
Just anecdotally, over some hundred interviews, I've never ever ran into a candidate who failed one of these FizzBuzz style loop writing tests because the didn't know the mod operator, or were nervous enough not to remember it.
I see a lot of candidates that can't remember what params some standard system call is supposed to take and I always tell them I'm happy as long as they define what they think it looks like. Heck, most of the time I have no clue myself.
I'm not claiming the candidate you describe (competent but can't remember a for loop) doesn't exist, just that I've never personally encountered such a beast. Quite the contrary, with a technically open minded interviewer it seems hard to get even the most introvert candidate to stop talking shop .. :-)