Comment by throwaway1979 | Hacker News Reader

Comment by throwaway1979 | Hacker News Reader

Parent Full thread

I'm really glad you mentioned "cute little puzzles". I am sick of them. I'm not really sure who has sufficient time to spend blazing the employer with feats of their programming prowess (I was going to say only undergrads would do that - however, many undergrads in top programs have little time to bathe).

If this was big companies being stupid, I'd get it. The HN crowd ... some of the smartest hackers around do little better sometimes. I was very interested in Stripe after hearing some of their folks speak (I don't mean to single them out ... they are a pretty amazing bunch from what I've seen) and saw the cute puzzles (https://stripe.com/jobs#engineer) as a listed way of impressing them. I immediately hit the back button and decided not to apply there. Perhaps it means we would not have been a good fit. Or perhaps, we need more respect in the recruiting process. I understand the need for fiz-buzz tests and all ... but things have gone a bit too far.

If a recruiter goes after you, and didn't bother to Google your name and look at your GitHub/OS contributions etc. he's obviously cold calling and wasting your time. He's the guy who asks DHH how much Rails experience he's got.

But, on the other hand, if you apply to a company, especially one that's more or less famous, puzzles are a very good way of separating the wheat from the chaff on a level playing field. Evaluating a strangers performance based on their solution to a common puzzle is very time efficient and allows the company to process large numbers of applicants fairly.

A common notion here is that great people can just pick up the phone, or will be actively courted by companies, so they don't need puzzles and recruiters. That is great for them (honestly, not a snark), but it's not scalable: Not everyone can know everyone, and there has to be a way for newcomers to enter into the network.

Case in point: Stripe - you want to work for them, but apparently you don't know anyone on the inside that can recommend you. And apparently you're not on their radar already. They do great stuff, they seem like an amazing bunch - so there are likely to be hundreds who want to work for them - and most of them, regrettably, are not qualified enough.

Do you really think it's fair to require Stripe to assign a member of their engineering team (that means you after they hire you) to review every strangers GitHub accounts, when you refuse to invest even a little time in signalling to them that you're the real deal (the puzzles aren't all multi-day endeavours)?

> Do you really think it's fair to require Stripe to assign a member of their engineering team (that means you after they hire you) to review every strangers GitHub accounts, when you refuse to invest even a little time in signalling to them that you're the real deal (the puzzles aren't all multi-day endeavours)?

Yes.

Then you're being unrealistic, because the expected value of a random applicant's ability is not that high.

But it works the other way around, too. The expected value of a random company is not that high unless the candidate considers themselves to have a good chance of success. Given how opaque the hiring process is from the point of view of an applicant, how do they know if it's worth spending their time solving a random puzzle?

Investing time in open source software is a better use of time from the point of view of a candidate, as they can show the same code to many different companies.

So it comes down to who is more in demand. Is your company so desirable that good programmers will invest a significant chunk of time just to have a chance with you? If the answer isn't an automatic 'Yes', maybe you should rethink your hiring strategy.

It also doesn't seem particularly hard to automatically filter out candidates with no GitHub account, or a GitHub account but few contributions.

Sure, I didn't say anything about that, but in this scenario it's implied that you're applying to this company, so presumably you know something about it and like it. You likely know a lot more about the company than it knows about you.

The Github filter is a bad one, though - lots of good programmers don't do the whole open source thing, and we need to stop this pretense of a good Github account being necessary for hiring. What started as a suggestion in some hiring blog posts has become overblown into dogma now.

At some point an employer needs to see code that a candidate has written, so unless the candidate wants to write code specifically for each company they send a resume to, maintaining some open source software is surely the most efficient mechanism to do that.

Artists maintain a portfolio of their work; why shouldn't programmers?

Artists don't usually have to show what brushes or paints they used to make a particular work.

Sometimes it's best to evaluate the work rather than what goes into it. Granted, this isn't always the case and can't be attributed to multi team environments, but I've often been asked for code when I have a large number of (successful) projects solely developed by me.

Also, small code samples are fine but some employers are in the habit of asking for major work (demos) just to show off your coding chops. That's bad.

That's because it usually doesn't matter how an artist creates a piece of work, as long as their methods are not too idiosyncratic; what matters is the end result.

For a developer, the how matters as much, or even more so than the result. You could create a complete mess that still manages to just about work, and it wouldn't be obvious from a cursory execution of the binary just how bad it was.

Ideally, if I was going to hire someone, I'd want to know how they develop a reasonably-sized project. An interview or puzzle can give me some idea, but a GitHub account is far superior because it would show me how they work over a longer time frame, and perhaps with other people. I honestly can't think of anything that comes close to demonstrating skill.

I understand the urge to minimize risk as much as possible. However, the truth is that in any field whether it's business development, sales, marketing, etc. what you do see most often is the person's track record and not how they actually went about doing things.

So the fact that you can actually see more for a programmer is actually a bonus, but that doesn't mean it should be abused like plenty of companies try and do these days.

Risk aversion is fine, and obviously you want to hire the best candidate but like mentioned elsewhere on this thread:

1. Not everyone has public github projects and I hardly believe anyone should have to work on open source stuff just to show off their coding skills.. Thats really not the right reason to work on open source stuff.

2. For web projects while you might not have access to backend code you can always look at the front end and see if the guy is any good. This is especially true for me considering the fact that I have multiple projects developed from start to finish live and available on the Internet.

Again, small demos or code sample requests are always fine. However, an entire project to get a job somewhere ? Might have been okay when I had nothing to show for it, but after a bit of experience on your resume, it gets old..

Do you really think it's fair to require Stripe to assign a member of their engineering team (that means you after they hire you) to review every strangers GitHub accounts, when you refuse to invest even a little time in signalling to them that you're the real deal (the puzzles aren't all multi-day endeavours)?

There's no such thing as fairness in the job market. If Stripe can't find enough programmers to fill their needs, then yes, they need to allocate their own labor to recruiting. At some point, the business cycle will swing the other way, and it will become necessary for programmers to put our time into hunting for companies who want to hire even one programmer.

C'est la vie.

> If Stripe can't find enough programmers to fill their needs...

... then allocating the programmers they have to sifting through a completely open "jobs" mailbox is still probably not a good idea.

> At some point, the business cycle will swing the other way, and it will become necessary for programmers to put our time into hunting for companies who want to hire even one programmer.

For the level of programmer competence we're talking about here, that hasn't happened yet.

For the level of programmer competence we're talking about here, that hasn't happened yet.

Define that level non-tautologically then, please.

I agree with you that cute puzzles are annoying and unnecessary, but if you look at the puzzles Stripe has on that page, you'll see that they are anything but cute puzzles. Something like the CTF that they built is an awesome and fun way to see if a developer has the chops to run with the big dogs. These are cool real-world problems. They're not sitting you down in front of a whiteboard and forcing you to complete esoteric puzzles in a limited amount of time, yet they're giving you a few different ideas as to how to impress them. You have all the time in the world and it's up to you to prove it. Stripe, to me, seems like an amazing company with some of the best talent out there. Just my two cents.

They are boring though. If I have to take a programming test, I'd rather take that time build a real feature onto your product, even if for free. If you like the results, at least I've created real value for my time, not just played a silly game. There's a risk that you won't like the results and it will be thrown away, but that's a reasonable gamble.

I think some of the ITA programming puzzles (for instance) are fun on their own merits.

Not to get too meta, but what if the company essentially buys your argument? "Yes, this is a non-real world problem for entertainment purposes only..."

But they want to find programmers who find such problems entertaining.

Filtering only goes so far. If you rely on these tactics instead of asking the actual questions. You will very soon end up hiring people who know nothing about the real world, but would have memorized tons of interview trivia.

The problem with puzzles is just that. You ask questions and hire candidates based on issues that have nothing related to the job. The end result is you up hiring the wrong people and then say you can't find good programmers.

You can't find good programmers because you are not asking questions meant to hire good programmers. You wanted puzzle experts and that is what you got.

Maybe we are talking past each other. This is the page I was thinking of:

http://itasoftware.com/careers/work-at-ita/hiring-puzzles.ht...

These aren't classic Microsoft "puzzle questions". They are programming challenges that say a lot about your approach to programs and problems. In particular, all of them ask you to make tradeoffs between correctness and performance.

They are cool enough to attract engineers, hard enough to discourage the idly curious, and unique enough that no published solution exists. That's why I said that someone who does this kind of puzzle for the sheer enjoyment of it might be exactly what a high-level software shop is looking for.

Would I want a "puzzle expert" who could solve the problems on ITA's page? Hell yes.

No way. I'm not going to do free work for your company to make money off in the hopes that you'll give me a job later.

What if you did the work for free, but the company would have to pay you for your work if they liked the results, even if they ultimately did not hire you?

Wouldn't that still be better than working on some contrived game that has no real-world relevance for free?

Then you aren't really working for free. More like on contract.

But that would only be true if they decide to use the results. Otherwise, it's just another contrived quiz with no real-world merit. The worst case is remains the same as the status quo, but the better case is that you'll be paid for your time in the interview.

It still seems like a positive step. No quizzes at all seems like an even better way, but I did qualify the point with the assumption that the quiz would happen regardless.

This is how we hire, and I think it's great. Both sides get to see if it's a good fit, we see how you work and if you have a product sense. You get to see if you like our product and company culture. Win-win.

As others here have mentioned, the questions on the jobs page are explicitly designed to not be cute puzzles. If you don't have a Github profile or other code samples, these questions help us get a sense of how you think about programming. But they're also entirely optional - we should perhaps make this clearer. We'd love to hear from you if you're still interested in Stripe.

Maybe I'm just incredibly naive, but are fizzbuzz tests really that necessary? I have a hard time believing anybody could get an interview, let alone graduate with a BS in compsci, without being able to do something THAT simple ....

Yes, yes they are. I don't even see all of the resumes submitted to my company since they are filtered through the recruiters and our HR guy but the ones I do see still need a fizzbuzz test more often than not.

Granted, it is not a thing you should give everyone. I can usually tell when I look at a resume that such a thing is likely to be pretty pointless (It points me to a github account I can look at instead? Why bother? The resume is clearly written by someone who knows what they're talking about and isn't stuffing everything they've ever done onto a page in a game of buzzword bingo? Probably safe.) so I don't give fizzbuzz tests to everyone but sometimes you have to be sure.

While this is a little disappointing in that it speaks to the quality (or lack thereof) of developers coming out of college, or elsewhere, it is also really encouraging too! the fizzbuzz problem is actually incredibly simple and I was able to write code out by hand that worked in my python interpreter.

Hopefully I'll have some good github stuff to point potential employers to soon (i'm working on some things now). But at least I know that I should be able to pass some of the incompetency gauges fairly easily!

also, i've never heard of buzzword bingo before, but it sounds interesting! maybe i'll make a program for buzzword bingo. You can use it in your interviews! :P

Well, not to burst your bubble, but fizzbuzz is a negative filter (ie passing it is the minimum needed to even be considered for a dev job).

Congratulations. You're better than 50% of the programmers I interview. Want a job in SF?

Yes, they are necessary. I've seen "senior" people bomb on it. There's a segment of "developers" that are good at selling themselves as programmers, manipulating the interviewer so you talk yourself into thinking they are awesome. You've probably met some of these types at conferences, and think they'd be a worthwhile hire. But stick a simple programming problem in front of them, and they can't back it up. if you're not that great at spotting these people, they can slip by the average interviewer without something concrete like a FizzBuzz to show they can back up their talk. And once they make it into your organization, if you don't catch them early, they can be extremely destructive.

I interviewed a 'senior' developer who had been a consultant for several years. When we asked how he implemented past projects, his answers were really vague. We got suspicious and dropped down to a simple problem like, "write a loop to print the numbers from one to ten."

The guy was writing some really weird stuff. When asked to explain, he confidently told us that in C++, every time you reference the iterator variable inside the loop, it gets increased. His solution was something like:

    for (int i = 1; i <= 10; i++)
    {
        printf("%d\n", i);   // <== i gets incremented!
        i--;  // need to compensate for the extra increment!
    }

(except I'm pretty sure he didn't get printf right)

So, yeah, I see FizzBuzz as a wonderful way to weed out the incompetent right away, allowing you to focus on deciding whether you need capable or exceptional.

Incidentally, one of the guys on my senior project team in college didn't understand how variable assignment worked. He would write:

    int flagsVar = COMMON_BASE_FLAGS;
    flagsVar = ADDITIONAL_FLAG_I_WANT;
    CallAPI(flagsVar);  // Why doesn't it work?!

He graduated.

You have clearly never had to interview people. Fizzbuzz is super basic and that's the point. It quickly separates the person who was around when programming happened and someone that actually did the programming.

You'd be surprised...

But actually, fizzbuzz is just the bottom level. You also want to do basic coding exercises for whatever specific area you are hiring for, such as:

- Set up an extremely basic web app that lets the user enter a name and echoes it back to them.

- Make a sprite move across the screen.

- Display a 3d cube with different colors on each side and make it rotate.

- Make it play some music.

- Aggregate the data in this simple database.

- Here's some naive producer-consumer code. Make it thread-safe.

- Debug this code that has memory issues.

- Critique this (flawed) design.

- Update the CSS to make this page look nicer.

Basically, you never want to get into a situation where you've hired someone who has managed to talk his way through life. It's far less headache to just use a few sanity checks on every potential hire.

I wonder if there's a harder equivalent to FizzBuzz — one that doesn't take any longer to explain or do, but requires a much higher level of skill to pass. One we used at http://airwave.com/ was "Write a Perl subroutine to determine whether a string is a valid dotted-quad IP address," which is something you can do in various reasonable ways in one to five lines of code.

There are plenty, and yours is a good example of one. In fact I think actual FizzBuzz is not used much in the wild any more as it's too well known and thus drillable.

Is that really so much harder? Somehow, I expect you'd get a lot of opaque regexes like:

  sub isIP {
    return (shift =~ m/^((\d{1,2}|1\d{2}|2[0-4]\d|25[0-5])\.){3}(\d{1,2}|1\d{2}|2[0-4]\d|25[0-5])$/)? 1 : 0;
  }

I think we did get someone solving it that way once, and modulo Unicode, it's arguably a reasonable solution. But there are other one-line solutions that are more transparent.

The Stanford CS Ph.D. candidate whose solution included a Perl function for converting decimal numbers of up to three digits to binary was rejected.

I failed to note in my earlier comment that yes, it is so much harder. Fizzbuzz requires that you write a loop, print some strings, and either maintain a couple of counters or use the modulo operator. Your solution requires:

- writing a subroutine

- understanding Perl parameter passing

- understanding Perl regexps, including =~, anchoring, alternatives, metacharacter escaping, character classes, and repeat counts;

- the ternary operator.

And then you could probe things like:

- Do they know how to fix the bug in your version where it incorrectly accepts an IP address with a newline appended to it?

- Do they know it would still work without the return and the ?1:0?

- Can they figure out that they could simplify and clarify the code substantially with qr?

- Do they understand why someone might prefer a non-regexp-based approach? :)

Reply on news.ycombinator.com