One hiring filter that works
chrisstucchio.com
chrisstucchio.com
If you give me a synthetic puzzle to solve I wouldn't bother applying to your organization. It's much better to craft a problem to the specific domain that your company is dealing with. Then you can test for both domain expertise and problem solving ability in one fell swoop with the extra bonus of it not seeming pointless.
† The important addendum on this is that you're not going to have the top level of folks sending out blind applications unless that person is in the process of making a lateral jump into another sub-field. And even then probably not. But if you give a top applicant a synthetic puzzle after your recruiting contacts them, expect a large portion of them to thumb their nose at you. Similar note on top folks, though this is mostly pedantry -- the top OSS projects aren't on GitHub and similar. The largest and most prestigious of the OSS projects host their repos themselves.
True, however if you're a contributor to a top OSS project, you'd tell me that when you apply, and send me an example of one of your contributions that you're particularly proud of (at least I hope you do, otherwise I won't know!)
Puzzles and challenges are useful when the candidate doesn't have any good examples of their work that they're at liberty to share with you. I'm very wary of hiring someone without seeing any of their work. We've been taking this approach for awhile now, and it's worked out well for us.
I'm actually more likely to apply if there is a puzzle. I've sent a few resumes out blindly, it usually goes into the black hole. If there is a puzzle, I figure my odds of getting a response (assuming I get it right) go way up.
Of course, if a correct solution results in a recruiter replying and asking me to submit to resumator, I'm probably ignoring your company in the future. (Yes, this happened to me once.)
The important point that I'm getting at, however, is that the dynamics of hiring shift dramatically as you move up the skill / connectedness scale. As you move towards the higher percentages potential employees are not competing for employers, but employers are competing for hires. And the dynamics of hiring are dramatically different at that point. If you're designing heuristics for hiring, you don't want to create a low-pass filter, and I believe that synthetic puzzles are exactly that.
The point of raganwald's post was on the surprisingly high cost of false-negatives. What I'm postulating is that your suggestion is just yet another way to reduce the false-positives rather than a way to reduce the false-negatives.
Good developers don't just go around submitting their resumes to every job in existence, and not filtering out false negatives was insufficient in and of itself to produce a good applicant pool. If doing something that produces false negatives (which, yes, are undesirable) nevertheless convinces enough developers to consider your company who never would have considered it otherwise, then it seems like the advantages have outweighed the disadvantages.
Obviously, an even better scenario would be to produce the same level of developer interest without also producing any false negatives. Perhaps by investing sufficient resources in networking or in making his company famous to other developers, he could accomplish this. This would still be a possibility to explore in the future.
But just because his tactic has not produced the best possible outcome does not mean it has not produced a positive one.
Hmm?
https://github.com/torvalds/linux
https://github.com/rails/rails
https://github.com/jquery/jquery
Linus copied the kernel tree to GitHub when the kernel git repo was compromised. It is not the main repo. Of the others, only Rails and JQuery could even be reasonably be called "top" projects (Bootstrap? Seriously, you list a 7 month old project with 2000 commits?) and even those are a stretch. Major projects are well into the hundreds of thousands of commits. It's ludicrous to even muse of Bootstrap being in the same league as e.g. Mozilla, WebKit, et al.
If you define success by the number of people who have actually checked out and built the code, Bootstrap is vastly more successful than Mozilla & WebKit.
So far, 2 of 4 companies I have applied to have had a coding pre-screen and 1 of 2 companies that I have worked for have had a coding pre-screen.
How to hire B players: Implement serious recruiting processes and plan to spend a lot of time on it. Get creative with hacks like OP's.
How to hire A players: Identify them through never ending networking, research, and keeping your finger on the pulse of the industry. You'll never find them the same way you find B and C players.
If you're doing that then it's possible they'll find you, if you're not then nothing you can do is going to help.
And the thing is Google, Microsoft, and GM also have some amazing teams simply though blind luck. The kind of thing where a few friends all end up working for a manager they like on some interesting little problem.
Some people have only worked on such teams and don't understand why so many projects fail, and others have never even seen them and wonder how complex software ever get's written. But, so few people have a truly understand the range from inspiring though dysfunctional people and teams out there that talk of A, B, and C coders / rock stars becomes almost meaningless.
PS: One of my favorite Real Programmers was a systems programmer for Texas Instruments. One day, he got a long distance call from a user whose system had crashed in the middle of saving some important work. Jim was able to repair the damage over the phone, getting the user to toggle in disk I/O instructions at the front panel, repairing system tables in hex, reading register contents back over the phone. http://www-personal.edfac.usyd.edu.au/staff/souters/Humour/R... or the modern equivalent http://bellard.org/jslinux/tech.html
One thing missing in your list: Have 'A' leaders.
He went one step farther: If you have B and C leaders in your org, it's basically all over because few organizations are capable of taking the measures to correct that problem.
Sounds like your answer is: When there are items that smart people want to outsource, they often go work for people stupider than them rather than hire people to take care of it.
Do I have that right?
No, I am looking at the logical consequence of "hire people smarter than yourself". If A is smarter than B, then B is less smart than A.
Therefore: "hire people smarter than yourself" implies hiring people who are willing to work for someone dumber.
Your boss can be smarter than you at his job, and successfully hire you, smarter than he is at your job. That's an odd way to word it, but the fact remains -- there are generally different skillsets and competencies involved, and you are looking to build or be a part of a team with complementary talents.
A leader is spending a fraction of the time hands on that a programmer is. Given the developments in technology and the additional time the developers are getting with it, if they're not better than their leader in a few years, you've got the wrong programmers.
Conversely, you've probably encountered highly intelligent people that have the people skills of a bug.
But software development specially in small teams doesn't work that way. Many a times, junior programmers require direction and even help in completing their tasks. So it comes down to two problems (three, third one later)
1. Either you give someone sub-standard work to maintain that you be able to help them. This will lead to demotivation eventually
2. You accept you are not in position to help them. This generally leads to friction and a bad boss will try to hide incompetence by architecture discussions, meetings etc
Smartness is subjective. Which leads to the third problem. How can you assess skills of someone better than you? You can only go as far as grilling a candidate to your own competence. So a B, C player will only be hiring other B, C players and that will soon degenerate the organization.
No one (ok... almost no one) has no management nor coding skill, rather each of us has a mix of the two. The great organizations I've worked in had extremely smart people in management and in development, but the type of brilliance was different.
So yes, hire someone smarter than yourself.
If you're a decent company, you shouldn't hesitate to invest in B,C players so that they become A players.
[1] - Of course I'm talking about humans who are mentally healthy.
Caveats:
[1] Implicit assumption that the person is actually interested in becoming better.
[2] I understand that this takes a "toll" in productivity because you could be out there building stuff instead of helping these people improve. However, once a company reaches a certain size, I should think there is enough "boring" work for the company to staff "B" players to handle those, while simultaneously making them spend another chunk of their time working on becoming better.
How to hire B players: Offer market value.
How to hire C players: Offer below market value.
You do two things when you offer more money.
1) You are more careful about who you hire. 2) You actually attract good people who would make a move for that kind of money.
I would love to hear an inverse argument against this.
So; offer an environment where people want to work. Give them interesting work. Add a dollop of patriotism.
Ultimately, if you had an interesting product, a great environment and offered extremely competitive salaries then you would most likely end up with interest from more people in the "A" range.
To use real world examples, no amount of money you offer Salman Khan now would pull him away from his non-profit Khan academy to go back to the hedge fund industry.
If you're going to point to your step 2, then that sounds like a tautology. "Attract good people who would make a move for more money. Offer them more money"
Isn't that what acquihiring is about though? Purchase the talent that wont make the move for money.
* It motivates B and C players more. So you get a lot more chaff to filter out in your application process. You also get many more D players (the ones who are knowingly incompetent and are actively trying to deceive you into recruiting them).
* The major motivation of A players usually isn't money (once you reach the "fair wage" stage). So you don't attract that many extra A players.
* Some of the A players who do get tempted by the extra cash discover that it's not quite as wonderful as they expect and leave for more interesting work. Wasting large chunks of your time and money.
* It may turn some A players off. There's a unwritten rule that you get more cash to make up for dull work. So offering silly money implies to some that the work is going to be exceptionally uninteresting.
The absolute best way I know to recruit A players is:
0) Understand the role that needs filling and the kind of person that needs to fill it
1) Have a really interesting job
2) Know how to access the A players
(I'm amazed at the number of organisations I come across that skip step 0...)
3) Be prepared to be truly market competitive in salary - your target A player may have a 20 year consistent track record of getting sh*t done, respect their sense of self worth.
Being immodest I consider myself an A player, when a startup offering "competitive salary and benefits" low-balls my current income by 50% they've wasted my time and theirs
This can't be emphasized enough. What you consider "silly money" probably isn't silly money to an A player. In fact they may even find it low.
Indeed I'd argue that it's an implicit part of performing step zero - understanding the role and the person. If you don't understand what your potential employee is looking for in a role - be it money, responsibilities, working environment, or whatever - then you're never going to find 'em.
For example I'm continually amazed at the number of companies who decide that they need an uber-geek with a ten year track record of delivering e-commerce platforms (or whatever) - and then go about recruiting them as if they were unemployed. Sigh.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well (but still below the .50 level, standing alone). One is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. Another is a general cognitive ability test (an IQ-like test, such as the Wonderlic personnel screening test). Each of these kinds of tests has about the same validity in screening applicants for jobs. Neither is perfect (both operate at about .4x level in validation studies), but both are better than anything else that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
For legal reasons in the United States (the same consideration does not apply in other countries), it is difficult to give job applicants a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of a hiring process. The Griggs v. Duke Power, 401 U.S. 424 (1971) case in the United States Supreme Court
http://scholar.google.com/scholar_case?case=8655598674229196...
held that cognitive ability tests used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring process had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Companies outside the United States are regulated by different laws.
Schmidt, F.L. & Hunter, J.E. (1998) The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings,” Psychological Bulletin,
http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...
via Google Scholar.
Yeah, I'd pretty much say that the IQ test was pointless in this case and luckily that was the only company I had run into using it.
EDIT: Forgot to mention that the test also took 3 hours to go through and had no limit on the questions that you would get. There was no possible way you could finish every question on the test.
Could someone do an (awesome) academic study which shows that all programmers benefit from g, or at least that all developers in x language do? Or add some specific problem where g is hugely useful (algorithm design?) and then make that problem a requirement for many programming positions?
There would be a huge benefit to technology hiring if we could start using a cheap, standard test result vs. relying on proxies like school admittance (where they can screen for IQ), or expensive work-sample tests (which burn more of the applicant's time, selecting against busy/good candidates).
I'd rather advantage poor, possibly non-US, high-IQ, busy applicants over either graduates of elite schools (correlated with rich) or people willing to spend the time to take a 4-40 hour unpaid assignment.
In that context, there is a peculiar and consistent gap between the best hiring strategy and the strategies institutions actually use.
For hiring faculty whose main focus is research, the best strategy is targeted attempts to hire superstars -- the research equivalent of a Guido van Rossum, say, or a Linus Torvalds, or someone near that level -- or early-career people where you have very good reason to be confident they will be future superstars.
It sounds obvious to say "go after superstars". But what's strange is that in any given field only a very short list of institutions actually do it. At most academic institutions it's surprisingly difficult to convince the institution to go after superstars.
I think the reason for this aversion is instinctive. If you're hiring it's much more comfortable to go the conventional route, advertising a position, interviewing a short-list and so on. This produces a situation where most of the interviewees want the job pretty badly. This gives the hiring committee a lot of power, which is a comfortable position to be in.
Unfortunately, it also produces lower-quality candidates than identifying the best people and going after them directly. A superstar will be interviewing you more than you are interviewing them. You'll need to get creative in convincing them that you have something to offer, something they can't find anywhere else. This is much less comfortable.
A useful heuristic: if most of your job offers are being accepted, your standards are way too low.
Focus on how good your company is on running an engineering department. Especially if you're a startup.
Do you have a VP of engineering who knows what the hell they're doing? Do you insulate the devs from sales people making ridiculous promises? Is your code base in good shape so someone can come in and make a difference without having to learn the complete spaghetti stack? Do you have a well designed deployment procedure that has dev, test, and production environments?
How do you approach scheduling? I'd say this is the number one issue that leads to bad experiences for developers, so tell us up front that you have this under control. Do I ever have to hear a business guy say "I don't see why that would take more than a few minutes..."
In other words, prove that every problem isn't solved with "developers work more hours."
Say something that at least implies you know that these issues exist. They are much more important than ping pong tables and calling each other ninjas. I bet you'll find plenty of pro developers who are sick of their current employer if you do.
Imagine if developers actually learned something about running a software company just from your job ad itself. That would be amazing. I would point to JoelOnSoftware as proof it would work.
But if you're throwing out maths puzzles when you actually want people to write Java business logic then I can see you ending up with the wrong people.
FizzBuzz is useful for finding the people who cannot even begin the problem; the people who couldn't craft even a vague approach in pseudocode. Some people can't sketch out a few loops and modulus tests.
You know that the person who isn't sure of the precise syntax of a modulus operator can pick it up. You also know that the person who is baffled by FizzBuzz probably will need too much hand-holding.
The message was that anybody who has the common sense to google an unknown string is qualified duh?
The first puzzle job ad is nice, but it quickly wears off, and would start to bring bored "oh, not again" feelings. And many people just aren't good at making puzzles.
It's the same as with paint-drawn NFS ad.
So please, when you hire people, have some humility. Know that your decisions are affecting someone's life in a major way, and you're probably making the wrong choice anyways, just do the best you can.
I already ask about 10 technical questions in each of the interviews I conduct, and having a problem online for each candidate to hack on would be really useful.
Now that I think about it, I may even consider using something like this to drive internal improvement projects. Some people just love the challenge, and the 'a' people always step up, you just have to ask.
This seems like a good way to only get good candidates but you may also have some very qualified people who don't want to play this game and find it unnecessary. In some ways its the equivalent of throwing out resumes with misspellings.
We did get one answer that was both incorrect, too slow, and massively over-engineered. I got emailed a multi-megabyte Rails project, boilerplate, tests for correctness and all. Bit It Didn't Solve The 20-line Problem. He was the guy we wrote the test to avoid!
The obvious downside is that competitors get your code too. But is there any credible analysis or evidence of how bad a risk this is? Because the risk of not finding good programmers is pretty bad.
I suspect that people are still way overestimating the value of source code in and of itself, relative to the value of the team that produces it. If so, that's a market inefficiency and open-source might provide an edge. It would also be more rational and satisfying from the point of view of the actual work. Imagine where mathematics would be if mathematicians still kept their proofs secret.
Can anyone point to examples that shed light on the effect of open-source on hiring?
Needless to say, we got one awful applicant after the other. Most could Fizzbuzz. Some could not. And others must have cheated their way through the phone interview - when you got them in person, it all fell apart.
One thing I noticed from the process is that little things like spelling mistakes, incorrect formatting, etc., in a resume was very telling. Without fail, the kind of person who had typos in his resume was the kind that failed our programming tests, miserably.
For example, if you are looking for a Front-End guy, why not look around web magazines, Web 2.0 products and services, Github, Dribbble and pick the guys you like and contact them.
http://haacked.com/archive/2004/07/12/google-mysterious-bill...
Like I posted in the "who's hiring", that doesn't work for me because it doesn't tell me whether you are what I'm looking for. What do you do? Why should I care about you?
So they weeded out the really bad applicants and as a result were left with so many good applicants that they're now in the same situation as the original article.
The key takeaway for me is not how people filter their applicants on real criteria, it's what they do when they have a large amount of seemingly equally skilled applicants. The original article demonstrated the 'unlucky' solution, I'm not sure what this article demonstrates since they still got too many applicants and they're spending time responding to all of them.
This article demonstrates a way to get a lot of highly skilled applicants-that's extremely useful.
The original article talks about finding 4-5 people worth talking to out of 100 people, 50 of whom can't fizzbuz.
We were in the situation of 20 interesting people out of 40 candidates, all of whom could fizzbuz.
If it were truly unmanageable, we'd probably have used weak Bayesian filters. E.g., degree, rockstar ninja, etc, all the things Raganwald originally advocated against using. I gather this is what Google and Facebook do.
One of my favorites from a few years ago was when reddit was hiring. One of the more fun aspects of it was:
git clone our public repo and tell us the most commonly used word
Another fun one was (I think? This was a few years ago):
find me a right triangle, the length of which's sides all add up to: $some_number.
[1]: In fact, this would be an awesome website if somebody hasn't made it yet: be a "recruiter", sortof, except you just post programming/math puzzles, we get to solve them, and then you get to match us up with employers.
-(void)fizzbuzz { int i = 1;int t = 3; int f = 5;int h = 101; int z = 0;
// here we do a for loop using i and incrementing it until it's equal to z
for (i = i; i < h; i++)
{
/* we check if both i/t is equal to z and if i/f is equal to z
if both are true we log fizzbuzz in the console */
if (i % t == z && i % f == z)
{
NSLog(@"FizzBuzz");
}
//check if i/t = z and log
else if (i % t == z)
{
NSLog(@"Fizz");
}
//check if i/f = z and log
else if (i % f == z)
{
NSLog(@"Buzz");
}
// if none of them are true we just act like we're learning to count
else
{
NSLog(@"%d", i);
}
}
}Does it work is the most important thing by a mile but if you've asked them to wrie production ready code then you've got to ask "how would I feel if one of my developers checked this into version control and how would it fair in a code review?"
Maintainability, readability, efficiency and most significantly the candidates ability to explain why they took the approach they did.
I wouldn't hold any approach against someone so long as they could talk intelligently about the trade offs and why they'd done what they'd done.
Even really simple problems are a chance to see whether someone thinks about things.
Your comment supposes that the loop will increment without interference. While you'd hope that was the case, it's not necessarily true.
Obviously with the loop as coded your comment is correct (that'll be the first condition that terminates the loop) but it could raise a question about whether comments should reflect the routine as written or just the section they're immediately referring to.
Or indeed whether comments for something so self evident are even useful or necessary... Something I'd definitely ask in an interview if I was presented with this code, not looking for a specific answer, just looking for some indication that the candidate had (or could come up with) some vaguely informed thoughts on the subject.
for(int i=1; i <= 100; i++)
printf((char *[]){"%d\n","Buzz\n","Fizz\n","FizzBuzz\n"}[((i%3==0)<<1)|((i%5)==0)], i);However, yes it is huge it's Objective C with whitespace what do you expect? ;)
Why they do it one way or another is an occasionally interesting follow up question.
1_,`0:'{:[~x!15;"FizzBuzz";~x!3;"Fizz";~x!5;"Buzz";$x],"\n"}'1+!100
[Edit: Found a way to shave off three chars]
Pertinent passage: FizzBuzz was presented as the lowest level of comprehension required to illustrate adequacy. There's no glory to be had in writing code that establishes a minimum level of competency. Even if you can write it in five different languages or in under 50 bytes of code.
This isn't even the Silicon Valley -- I live in Europe. Sometimes I wonder if I'm smart or not. I know I probably wouldn't be able to answer all of these trick questions I hear about. I think I'm much better at abstracting concepts and seeing the whole picture. I love programming, but so far I haven't found a place where I can have as much fun as when I am creating software on my own, free of constraints and in control of everything. When I work on my own, every now and then I stop coding, sometimes for days, thinking, sketching, until I know exactly what I'm going to do and can explain why in plain English.
Anyways, I think younger programmers might enjoy taking time answering these kinds of question, but I'm the kind of person that wishes he didn't have to sleep so he could spend more time working on his own projects, so I don't bother. If you want me to be interested, you'd have to find a way to convince me that I would be working in an environment that feels like I'm working on my own project. I want to work with motivated people that care more about their product than getting funded. I don't want to work in an environment where the number of closed tickets is a metric for productivity.
- The ability to perform on audition day doesn't necessarily correlate with the abilty to perform consistently in concert settings. - Great solo players aren't always great ensemble players. - It is very difficult to predict how an auditionee will mature over time.
That said, our field has yet to come up with anything better, and I think the organization I work for has developed a particularly good screening process:
On audition day, applicants are asked to perform some excerpts from literature our organization plays regularly, a prepared solo selection, and to do some sight-reading. This is pretty standard stuff for the clasical music industry and this preliminary round is done from behind a screen to help combat biases on the part of the panel. The excerpts are important because that's closest to the work applicants will be doing on a day to day basis if selected. The prepared solo helps show the candidates potential to excel in solo situations. The sight reading is crucial because it is a basic and necessary skill and correlates very highly with the applicant's quality of musical training.
The second round (generally 5-15 applicants from the initial pool of about 100) is where our organization does things a little differently. It consists of more excerpts and sight-reading, but this time applicants are asked to make adjustments to how they performed the selections (a little faster or slower, a little louder or softer, etc.). Observing how the candidates respond to these requests shows how nimble the musicians will be in a rehearsal setting, and also shows how well-prepared they are and how well they will take direction.
In the final round (generally 2-5 candidates), applicants are asked to perform alongside members of our organization. This demonstrates more responsiveness, how well the applicant listens, and how well the candidate performs in a group setting.
With that in mind, I often wonder why those looking to hire developers don't use a multi-stage "audition" process.
Step one could consist of some thing along the lines of "Here's a piece of code like what you might see in our organization. There's a bug in it. Find and fix the bug and send the revised code back to us." Step one could also consist of a short puzzle-type problem. Step one could be 100% automatic and would require the same amount of time for 100 applicants as it would for 1000 applicants.
Step two could be where applicants are asked to show samples of something they've coded in the past. This could also be where they are asked to perform a more difficult bug fix and a more difficult puzzle. In this round, applicants are told that their code will be read for documentation quality and use of good style.
Finally a final round could be done in-person where an applicant is asked to solve a problem with existing members of the team. This could be done in conjunction with one on one interview. This would help tell if the candidate is someone the team actually would enjoy working with, and how the candidate would be able to contribute.
I know I wouldn't mind submitting to such an application process because I could see how each step is relevant to what the company is looking for. It also shows that the company takes hiring seriously. Hiring is supposed to be hard. Knowing that every one else on the team has been able to excel in those circumstances gives me greater confidence that I'd be joining a great team.
As a result, the consequences for a bad hire are extremely high. Many orchestras have the audition winner perform with the group for a probationary period of a season to help guard against this. That said, I read a lot on HN about the consequences of a bad hire for a an early team member, so although the timeframes are different, the fear is similar.
As you say, it's all about making the relevance of the process obvious to the candidate. Also, companies should be as accomodating as possible in terms of timing of appointments, etc.
Interviewing is a two-way street. While you're trying to figure out my coding skills, I'm trying to figure out if I'm interested in what your company does and if I'd like to work with the people there. Personally if I knew I'll have to spend an hour of my time on interview challenges just for the chance to get some face-time with an engineer, you'd never even see my resume.
I don't remember what first brought me to HN, but I stay to learn about open-source projects I wouldn't otherwise find out about. The next version of my archives site is built with Flask and Twitter Bootstrap, both of which I learned about here. Also, my transcoding scripts use Kenneth Reitz's Envoy for the system calls.
Also because I realize that as much as I love music, my decision to become a recording engineer was probably ill-advised. There are days I strongly consider giving it up to move to the Bay Area and try developing full-time, hopefully at a startup.
Add: to me writing code is an activity that can be enjoyed professionally, unlike music.
http://www.chrisstucchio.com/blog/2012/leaving_academia.html
But I'd suggest that except in special cases, posts with titles like mine are probably not worth reading if you are uninterested in hiring or finding a job.
I am apparently puzzle obsessed today. Slow day at work.
I have no idea if they are currently hiring, so please don't bother them with job applications unless you see a job ad.
((First Row and Fifth Row are not well defined, but that's not particularly relevant to the problem.))
I often look through job ads out of curiosity, and it's remarkable how generic most of them are. Maybe they promise "high profile clients" or "working with exciting technologies". Maybe they even promise "a friendly atmosphere". I don't understand why anybody would answer such a job ad.
I'd be interested how many well qualified applicants they would get for a job ad that actually described what they are looking for: "We look for someone to work on the back end of our fizz buzz cloud service. The task will involve factoring numbers and counting digits."
I find solving puzzles at codeeval much more interesting than crafting custom cover letters, and the ease of applying means I'll more likely apply to a company there than elsewhere.
OTOH I don't think I'd classify myself as an "A" player, at least not yet. The more I learn the more I find I need to learn &tc.
(haven't heard back for anyone yet though. Probably a bunch of good applicants inundating their hiring team as well.)
Any thoughts/feedback welcome.
i mostly did the puzzle because it was a fun distraction.
another thing is that it self selects for people who consider problem solving fun. its not necessarily the most predictive of job performance, but it is better than nothing and you'll also be filtering for personality (no opinion on whether that's a good or bad thing).
sure it might weed out some good people, but it probably improves signal/noise ratio.
The point is not that everyone who can program FizzBuzz is talented. The point is that programmers who can't solve FizzBuzz are probably extremely untalented.
I'm getting to the point where I'm realizing that I'm only going to grow as a software engineer if I'm extremely selective about the work I take. I'd go into management before I'd work in Java or C++ full-time. (Of course, I'll use these languages if small projects require it, but if you're a "Java shop" I'm not interested.)
I wrote about this effect here: http://michaelochurch.wordpress.com/2012/01/24/how-any-softw... . It's long, but the TL;DR of it is that it's very easy for a company to become good at hiring if it gets its core technologies right.
A good way to do this is a humans.txt, whose purpose is not just to credit the developers but also the technologies in use. It's funny that a lot of companies are crazy about hiring right now, but haven't taken 30 minutes to add this simple file to their web server.
I suspect that statement is more absolute than you intend it to be. Would you not work on a new language if the compiler for that language generated C++, which meant that the language's runtime was implemented in C++? Which, of course, implies that you will have to write a decent amount of C++ code.
If you would work on such a project, then I suspect you mean "If the problem and solution itself are not novel, then I have no interest in using Java and C++."
My experience is that language adjustment is up to a 0.5-point drop (the scale is here: http://michaelochurch.wordpress.com/2012/01/26/the-trajector... ) in an engineer's ability. This is on a 3.0 scale where most engineers are between 1.0 and 2.0, so that's a huge change. Compensation tends to be linear in proficiency but exponential in value-add. 0.5 points is about $50,000 in market pay but a 2-4x (depending on the type of company) difference in productivity.
Sometimes it is worth the drop, such as when one's learning a dramatically new and more effective approach, or solving a fresh class of problem (e.g. real-time software, which is done in C). That isn't the case with enterprise, "pattern"-afflicted Java. Instead of solving the problem appropriately and well because I know how to do it, I have solve problems inappropriately and badly because I don't know the crappy idioms that people invented because they don't have appropriate tools.
On that scale above, I put myself around a 1.7. If I set foot in a typical C++ or Java shop, I'd drop to 1.2 or 1.3. Which means I'd risk falling to the level of a junior programmer and being typecast that way. Which means I wouldn't get the benefit of the doubt if I say, "This project wouldn't be complete shit if we'd done it in an FP language" (in some nicer words, of course). Your first 6 months determine whether you have technical credibility or not. If you don't, you can't blame the tools, even if bad tools are (as they often are) the cause of the problem. The unenlightened tend to think of all tools as equally good (all languages are Turing complete). No, not so. A 3x difference in CPU performance may not matter for most problems, but the 3x difference in human performance that hinges on whether one uses the wrong tools certainly does.
The drop to 1.2 is something I'd tolerate if moving to something genuinely interesting, knowing it'd only take a year to pop back up to the same or even higher level. An example might be high-performance C++ in weather forecasting, bioinformatics, or artificial intelligence. I'd gladly become a junior again for a year for that kind of stuff. On the other hand, most enterprise Java/C++ bullshit isn't really worth learning, not for a creature that has 60 years left to live instead of 600,000 (#ShitYouCantSayAtWork).
So yes, my language bigotry is more directed at the typical C++ or Java job than at the language itself.
I assume my reaction to C++ is that way because my C++ experience is entirely in embedded software / consumer electronics (Strongly C-driven but way more C++ than you might imagine).
It turns out that if you implement a bunch of Callables and ExecutorServices with LinkedBlockingQueues or SynchronousQueues.. it's basically functional programming. Don't get too hung up on the syntax.
You're talking about at least 3 distinct models: direct thread management, with improved locking routines (java.util.concurrent, see recent Prismatic thread on HN), actors/green threads (Akka and erlang BEAM), and STM/guarded read/writes (clojure, and you should read about Microsoft's attempt to graft STM onto C#, Joe Duffy's blog)
Actors and STM are great thought experiments but real-world production stuff tends to always circumvent them for the java.util.concurrent stuff.
Its a mixed picture, if you want to be honest. There are places like bofa where I've pushed into production a cvar processing module on 500,000 loans which uses only actors. No low level tinkering with util.concurrent, no Threadpool setting, no Executors, no synchronized anywhere in my code. In the past at GS, I wrote concurrent code where I explicitly specified the number of threads & we had a ThreadPool manager & internally it used the util.concurrent classes. At LinkedIn, I believe they started with actors & switched parts of it to Doug Lea's framework ( which was the sort of the foundation of util.concurrent). I have seen financial shops where they do half & half - the quants seem to like actors but the swing programmers like to have control on threadcounts, & they tend to do things like naming threads & creating parent-child relationships so swing can paint reliably & they can debug threads by name when it doesn't. I did the same in the past when most of my work was on the swing side, creating custom blotters to show trade tickets & the like. But now I primarily do quant model building work with actors & no more util.concurrent stuff. If you want to construct a lattice model to price structured options on a 100 different strikes per month for 5 years in advance, you can find no better friend than an actor. Actors are certainly not a thought experiment, very much a reality.
For why, I'd assume that they found the abstractions were getting in their way. java.util.concurrent is sort of the minimum abstraction (IMO) between nicer constructs and OS-level threads.
The platform is excellent technology. The language is mind-numbingly boring.
When Scala and Clojure projects go "straight to Java", it's generally for specialized purposes (where those languages don't have native maturity) or in already-discovered bottlenecks. I think some degree of that's inevitable. Like database tuning, it's just something you have to do when you get to a certain point where (a) your problem is well-defined, and (b) shaving 20% off latency is worth doing. I consider these good problems to have. If I have to rewrite a class in Java for performance reasons, fine. I'd never start a multi-thousand-line project in Java, though.
Also, I really don't think anyone mature minds writing code in underpowered or difficult languages. C++, I find, is actually a lot of fun to write. It's reading other peoples' code, in large amounts, that becomes intolerable in these languages. If I have to spend a day to answer a simple question like "how do I initialize this object?", that's bad.
Interesting fact: Closures were cut from the 1.0 release for time. Gosling wanted them in, but Java's original purpose was to run in machines like automatic coffee pots and TV "set top boxes", so they were far from essential for that domain. The folk history (which I once believed) of Java being deliberately underpowered so that teams of hundreds of mediocre programmers could be stapled together in it is false. It was, however, co-opted by the "let's commoditize developers" crowd in the late '90s, and we've had to deal with FactoryVisitorDelegates ever since.
from my time at sun, i remember speeches by Gosling that included microwaves and refrigerators ( i vaguely recall an example of the fridge telling you via java applet that you had run out of milk ). the hotjava thing ( web browser written entirely in java http://en.wikipedia.org/wiki/HotJava) created some anticipation around Word & Photoshop & Powerpoint written in java, which were brought up when Sun hosted a gigantic industry conference in Vegas. The Java Chip ( picojava http://www.jopdesign.com/perf.jsp ) was supposed to revolutionize the industry by running java bytecode natively. Honestly I think the dumbing down began once EJB's crept in. There was some nasty visual shit called the Beantool which automatically inserted getters & setters by mandating every EJB to be a Bean, and from then on things got downhill really fast.
Mine had already been primed with XML "XSLT can map anything to anything!", and Enterprise Java was just more of the same.
It's funny that Javascript and WebGL are cleaning up in the browser, and Java applets are almost nowhere to be seen.
You know how awesome that type inference is in Scala, and how stupid and annoying it is in Java to type stuff like Map<K,V> myMap = new HashMap<K,V>? I totally agree with how annoying that is. Here's a related example in scala.
def myThing = something.getSomeThing()
What is the type of myThing?
Java has weaknesses for sure but reading code is actually one of its strengths, precisely because of the boilerplate. Note that I'm not defending EJB or any other atrocities here, just the core language.
Lastly, I agree that syntactically-nicer closures would be better for Java. But it already has closures,
final Something variableImClosingAround;
Runnable myClosure = new Runnable() { public void run() { variableImClosingAround.doSomething(); }};
exec.submit(myClosure);
scala> (1 to 10).par.foreach(x=>print(x*x+","))
4,9,1,49,36,16,81,25,64,100,
For that problem, I'd just do it serially, because it's only 10 numbers.
Oh, it was an example for really big sets of numbers? Like really really big? Then I'd skip the .par.foreach and deal with java.util.concurrent directly, because it's worth my while if I have a serious problem worth optimizing.
"A newbie in concurrency whose mental model doesn't incorporate thread pools or executor services or synchronous queues can progress leaps & bounds in scala vs java when it comes to concurrency."
A newbie in concurrency needs to understand how many threads are in their threadpool, or else they should just do it serially. I'm not talking rocket science here, it's just a single integer argument.
Tried that. Now I'm back coding Java, Groovy, JS and looking into Scala.
The perfect management role for me is one where I can shape product direction, mentor some younger guys, and still code. The problem is that in all the companies I've seen, management ends up being I sit in meetings all day and get completely cut off from coding or even having deeper tech discussions. I just don't feel like I'm still creating at that point.
I think it's the 2 surfers outpaddling the shark scenario: you don't have to be great at hiring, just slightly better than other companies in your demographic(geographic and "itch yoru scratching". You do have to be great at dev retention.
Not that rare, actually. I have a hard time finding companies that want to use Scala.
Manageable parallelism is nice, though.
Regarding ArrayBuffer: It is not as easy as it looks like, because of the conflicting nature of Arrays vs. Generics (yet another JVM problem). Amusingly I tried using Manifests to do exactly that last week. I hit the wall because the Manifests seemed to spread arbitrarily. Will try with TypeTags/macros when they become available.
in order of confidence, here is how you hire someone for the job. (perceived)
100.000% - The job is done already, just like something you already had the same guy do and it was perfect, now you pay him.
95% - The job is done to your satisfaction, you know the guy and he's solvent and vouches for what he's done. He hasn't done just this, but maybe something similar for you. You're not sure if this works as advertised yet...
90% - The job is done to your satisfaction, you know the guy and he vouches for what he's done. He hasn't done this for you but did it for someone else you know, who vouches for the guy.
85% - The job is done to your satisfaction, the guy comes recommended only by people you don't know, but warrantees what he's offering and seems like he cares about his reputation and about you, seems solvent and in a situation to fix it if it's not what you thought you were getting.
80% - The job is NOT done (none of it is), but you know the guy (rest per 100.000%)
75% - The job is NOT done (none of it is), but you know the guy (rest per 95%)
70% - the job is NOT done (none of it is), but you ... (rest per 90%)
65% - the job is NOT done (none of it is), but... (rest per 85%)
60% --> the job is not done and you have a guy offering to do it who comes recommended by people you don't know, you haven't worked with him, and he doesn't seem like he's in a situation to warrantee what he's going to do or care utmost about his reputation and about you. However, he brings proof he's done it before (portfolio).
55% - the guy comes with recommendations from people you don't know, but brings proof only of things OTHER than what you want him to do for you. He is not in a position to guarantee what he's going to do for you.
50% - the guy comes with recommendations from people you don't know, and doesn't bring proof of anything he's done.
BINGO. This is where you are if you get a stellar resume and cover letter with references from people you don't know.
Now of these resumes, let's go through the hiring filters that work... 40% of the time... 30% of the time... 20% of the time... 10% of the time...
Ready? Okay, you know what, let's not go through all that. If you are trying to filter the hires so that you're getting the best of it like this, well, what can I say. Good luck pal.
C programmers can't do FizzBuzz[0]. B programmers can, and LOVE PUZZLES. A developers are the ones whose idea of a "puzzle" is solving the fundamental business problem via software. B programmers think that super elegant engineering is great, and don't care about business so much.
These B players are also the ones who often will torpedo a candidate by asking them trick questions (trick programming questions, irrelevant or obscure interview questions, etc.)[1] These questions are great for making the B player feel superior. Puzzles are a good way of doing this. This way if the candidate is hired, the B player's ego is mollified by knowing he was able to trick the candidate during the interview, and if the B player decides the candidate isn't deferential, he can claim he doesn't have the engineering chops for the job.
I sometimes wonder how many of these complaints about people "not being able to do FizzBuzz" are from these B players. (I've never used FizzBuzz as an interview question, I don't ask programming questions, but I've never managed to hire someone who couldn't do FizzBuzz for a programming role.)
At any rate, one of the things that makes interviewing at a given place a total roll of the dice is when you see evidence that they don't know what they're doing. An HR person trying to screen you who doesn't know the difference between java and javascript, for instance.
But another signal to me, is these puzzles. These puzzles scream to me "This company is run by B players!"
For instance, this is how I knew Google was not a company I wanted to interview with (even though they pursued me relentlessly for about 6 months, and still pester me periodically)... before I had the inside info that confirmed what the puzzles indicate.
I also don't believe in the "A people hire A people and B people hire C people" mantra, because I worked at Amazon (A C organization if there ever was one) and Bezos liked to say this (a D manager, in my book... but obviously he thought he was an A.)
I'm sure to some people, I'm a B player, or a C player (certainly the puzzle fanatics might think that) but I have seen a lot of companies get lost spending all their time working on clever engineering and not solving business problems. I like engineering problems and solving them, but the business problems are the really interesting ones-- its amazing what software can do to solve these problems, and often, non-engineering management has no way to know, which is why specs often don't work because thy ask for only what they think is possible.
So, I'm not being a snob here. I'm just saying- I use these puzzles as a filter. I wouldn't apply to such a company (if I was still working for others.)
[0] Theoretically, anyway. I'm a bit dubious about this claim-- all the programmers that I've seen who were a bad fit and "couldn't' do the work" were actually good programmers, just dealing with bad management. I have met a lot of non-programmers who think they are programmers and who only can do things that fit their pre-determined ways, e.g.: They're trained in a set of foundation classes and can't solve problems without using those classes (which have already solved things like sort, etc.) I assume these are the people we hear about when someone says "can't program fizz buzz".
[1] For example, once I was interviewing for an internet infrastructure job and got quizzed on Win32 API details (like "what are the parameters for this obscure graphics call?" "I don't know, haven't looked at it in years." "It says here on your resume that you know Win32!". Similar example- interviewing for a Mac development position in the PowerPC era, mentioned I'd done assembly in Z80 back in the day and had a little familiarity with 8086 assembly-- getting questions about the Pentium instruction set. I could answer broad questions, because I had a high level exposure, but I only claimed to study 8086 back in the day, not be writing assembly currently, and it was for a Mac development job, and he wasn't even talking about the PowerPC.
Can you expand on this? Bezos seems to be doing a spectacular job at Amazon.
I've heard bad things about working for them, but as a consumer, I love Amazon.
AWS has gotten a lot of hype, as has the kindle, but I don't think either of them are ahead of their competition. My point though, was internally.
Amazon is full of B and C players who think they are A players, and that goes all the way to the top. (There are some A players there too, of course, but they often leave within a year or two.)
I couldn't agree with this more, although I would say "B programmers love to solve every problem. A programmers love to solve the problems that matter."
I have worked with many people, some quite brilliant, that simply can't prioritize issues. If they perceive something as broken or sub-optimal, they absolutely must fix it, no matter how minor the problem. If you can reign them in and keep them focused, this type of programmer can produce great things. But it takes a lot of management time and energy. I'd rather work with someone who can see the larger picture and focus on solving the actual problem, even if the implementation isn't 100% perfect.
Any criteria that says Google doesn't hire A players is a joke, or completely meaningless.
i also enjoy puzzles, and would probably be one of the people who solved the puzzle in the interview screening test just because it was a fun thing to do. i would never, however, use tricks or puzzles in an actual interview, and my focus is strongly on encouraging the candidate to put his or her best foot forward, and supporting them while they do so. an interview that lets me get a feel for the best the candidate is capable of is far more valuable than seeing who does and doesn't get tripped up by some random trick or either-you-get-it-or-you-don't puzzle.
and finally, i now work at google, and my interview was pretty straightforward algorithm problems - no "how would you move mount fuji" conundrums or random rec.puzzles-style brainteasers. i'm not sure if google actually did use that sort of question at one point, but they certainly didn't last year. (anecdotal evidence, yes, but i had six different interviewers, and surely at least one of them would have trotted out a puzzle if it was still a prevalent practice).
In other words, this was a marketing/PR win - not a win in terms of how you are asking people to apply.