ITA's old “Hiring Puzzles” is still a treasure trove of good problems
web.archive.org
web.archive.org
These appeared during the post-dotcom bust pre-startup boom period of 2004-2008. If you look at CS enrollment rates of the time you’ll see they were at record lows. Software paid well, but not that well and most engineering jobs were really bad ones resembling Dilbert comics. The culture of treating engineers well, giving them good tools, snacks, unlimited pto didn’t happen until later.
At the time people solving hard problems and keeping up with changes in programming languages and tech were doing it because they loved it. Nearly everyone doing tech was doing it because they where obsessed and passionate about it. This is because if you weren’t there where much more pleasant ways to make about the same money.
These ITA puzzles were a way of signaling “hey! we’re a company of people that love to code and work on hard puzzles too! If you’re the type of person that sees puzzles like this and can’t get it out of your head for the rest of the day, you should check us out!”
These puzzles were not a leetcode like gates (in fact they very explicitly were NOT required for applicants), but rather a way of advertising to programmers that ITA was not just another boring big corp that would suck your passion for coding away.
-- Dave, ITA cofounder and creator of some of the puzzles, including my personal favorite, “Rebus”.
However, from the linked page:
> To be considered for an interview, you may need to solve one of the programming puzzles below.
In fairness, they used the word "may". But I wouldn't call that "explicitly NOT required".
These are fun puzzles but terrible interview questions.
"When implementing the server, aim for scalability and robustness. (Many submissions fail due to lack of robustness!) Your submission should include a description of the steps you took towards those two goals. Keep in mind that the client may be buggy, or even malicious. For example, if a client connects to the server and sends an infinite stream of the byte 'X' with no line break, the server should deal with this case gracefully. Please do not use an existing networking framework (e.g., Twisted or asyncore for Python, ACE for C++, etc.) to implement the server."
Not only no, but fuck no. I settled for a job in the IT section at a bank working with COBOL and EJBs because of this shit, until I'd finally had enough, co-founded a company, moved to San Francisco and implemented a SANE hiring process that treated candidates like professionals. Most of us did, so I guess in a way these horrible and misguided hiring practices were partly responsible for the next resurgence, because we'd finally had enough.
Please do not use an existing networking framework (e.g., Twisted or asyncore for Python, ACE for C++, etc.) to implement the server.
...would be a massively positive signal to me as a candidate. Places that truly value "full stack" abilities like this seem to be hard to track down.
If I want to see code, I'll tell them beforehand that I want to build a little project together with them (less than 50 LOC) that does this one trivial thing, and that they can write beforehand if they want to. Then when they come in, we either build it together if they haven't done it beforehand, or we change the specs if they did. And no, no bonus points for doing it beforehand. That's just so that they can feel more comfortable in the codebase if they feel it's necessary (being nervous in the interview and all). I'm most likely not going to compare their code to others; I'm only interested in whether their code meets the company standards.
I've also fired many people. It's not hard to do, nor does it cost much.
So many of these recruitment processes are predicated on the principle of "what if someone lies to us?" for fear of hiring the dud. But in reality, it's so hard to bullshit someone knowledgeable that the actual risks are tiny.
Yes, I've encountered people who couldn't code. I've had such resumes passed to me by people (usually non-technical, or recruiters) telling me how awesome this person is (because they don't know enough about the subject matter to tell the difference). But it takes very little conversation about any topic one is knowledgeable in to sniff out someone's bullshit.
Think of it this way: You have a leak in your house and hire a plumber. The guy comes and tells you that you need this and that. How do you know he's telling the truth? Now imagine that you're a plumber who's been asked by a friend to sit in on the conversation. How long do you think it would take before you can confidently call bullshit on what he's saying?
If YOU know your shit, you can tell when other people don't. And if you DON'T know your shit, you're going to have a hard time no matter how much code they throw at you.
I'm in Europe, so it's not easy to fire wrong hires after trial period, but I have never even seen anyone admit that they made a wrong hiring decision.
As an interesting case, one company I worked at hired an architect who had failed their rather trivial coding test but interviewed well. That person, together with company management, ended up driving away several competent engineers.
I won't work any more for employers who don't require a programming test. Your experience may be different.
It has also been my experience (20+ years) that sociopaths and egotists (the ones who will try to bullshit you beyond a few ham-fisted attempts) are in such a minority that they're not worth basing your interview process around.
The only cases of hiring engineers who couldn't code that I've ever heard of directly from colleagues over my entire career (that weren't third-hand re-tellings) have been where he either was not interviewed at all, was only interviewed by someone non-technical, or was hired by someone non-technical over the objections of someone technical.
TBH I think this is a manufactured problem.
On Wall Street everybody uses coding puzzles to screen candidates. There is really no other way.
But the online coding-puzzle sites they use are mostly really terrible.
For me, these puzzles were what got me to change careers; at that point I was in a non-technical job and didn't have any CS training. Still, I saw these puzzles and was hooked. Figuring out how to even set up a basic coding environment was part of the puzzle for me.
I spent a few weeks on "Strawberry Fields", eventually getting to a brute force solution that passed all the tests. I was about to send it in, and happened to meet one of the ITA folks at a social event. After a few minutes of talking to this gentleman, I realized he was in a completely different intellectual class from me, and that I had to do a lot better than hacking together a solution to a single optimization problem if I really wanted to work in this industry.
I persisted. I kept solving whatever coding problems I could find and rung by rung, climbed my way into some interesting and challenging software and data roles. I bombed plenty of whiteboard interviews along the way.
It sounds funny to write down, but it is not an understatement to say that those ads on the T changed my life!
I don’t see the question I remember pondering, which was simpler than these but more fun to think through on a short subway commute— something like “Create the longest possible string of connected movie names, like ‘Independence Day of the Dead Man Walking...’”
Edit: I found it! It was called “Sling Blade Runner” and it ran from 2007-2008 so my memory was off by a year. Solution: https://github.com/vy/ita-puzzles/blob/master/README.md#slin...
So much stuff on the internet (and business in general) thinks that "hey, it worked for someone - it must be a new paradigm!"
The people who wrote those puzzles would never think of writing them as incentives to join their company now - because that idea has long-since sailed.
So much of the crap part of hiring (and business in general) is because it involves taking something unique and interesting that someone has done, and rehashing it until it's a hollow, shallow, uncreative version of its original effect!
I eventually got an offer, but chose another job. They were acquired by Google soon after, and I sometimes regret not taking it!
You have to be kidding me. It also is biased against people whose time is spent playing video games or who do nothing at all.
Some jobs are meant for people at a certain point in their career. Sometimes you don't get to have everything at the same time.
People like you would argue it would be better to shut down hospitals than let doctors work long hours because those long hours might make women not want to do the job.
Some jobs are meant for people at a certain point in their career.
That should be determined by their employability, not the state of their personal life.
People like you would argue
No they wouldn’t; strawpeople don’t argue anything, they just fall over.
Isn't that the point of a hiring process?
The point of a hiring process is to exclude candidates that are legitimately not suitable for the role.
A reasonable contextual interpretation of the grandparent comment is that the hiring process will end up excluding suitable candidates for illegitimate reasons.
Whether they should have to slavishly spell this out in a message board comment is a hot topic.
- It was definitely fun working one's way to solve these
- And they are nothing like the LC problems thrown around in interviews nowadays
- Neither are these ITA problems a good fit for a 45 min interview from the duration perspective or getting `the right answer` perspective
[0] https://www.deepaksurti.com/blog/category/ita
edit: added the page link where I posted my solutions
The thing about the puzzles was that, once you start thinking about one, you want to see what a solution would look like, and the only way to find out is to solve it. Having solved it, you want to know how it compares to others'. So you send it off, even if you aren't really interested in moving cross-country.
In my case, the puzzle was "add-a-gram": given a word list, find the longest sequence of words in which each word after the first is an anagram of the previous word, with one letter added.
I thought about it for a day or two until I realized I had a workable strategy. Then I sat down at my laptop, checked the clock, and started coding. Exactly two hours later, I had a program that, after reading in the dictionary, was a one-page, deeply-nested loop that exited when it found the answer.
It ran in ten seconds. How could I know if that was any good? I sent it off to ITA. The next day I got a call. We talked a little about setting up a phone screen, but she let slip that their solution was faster than mine. That could not stand. I spent the day inverting the control flow, with memoizing, and got a 10x speedup for less than 2x as much code, and sent that off.
I went on to interview. At the interview, they assigned a programming task that was a simplified version of the puzzle. I worked there for three years, until I got fired for being depressed. (I found treatment five years later.) I don't remember much of it because I was sleep-deprived, from having toddlers who would only sleep in the daytime. But I encountered there the most intelligent people I have ever known. Most were musicians, on the side.
I had been there three months before I thought to check their solution. My second version was faster. (Bitwise operations FTW!)
While there, I encountered a problem that I have used in conducting dozens -- maybe over a hundred -- interviews: Undergraduates solve it in two minutes, graduates in five, MSs in 20 or more, PhDs either never, or as fast as an undergrad. Every single person who has solved it used the same hand gesture in describing the solution.
Sometime later, ITA were bought up by Google, and I got some cash for stock. Taking care to owe only long-term capital gains did no good because Google paid cash, so we all ended up owing "alternative minimum" taxes as if we had made all the money in one day, not over N years. (The same thing happened when my next employer was bought by IBM.) Alternative minimum taxation is a terrible thing.
The version of the puzzle I was assigned during the interview is one we sent to candidates between phone screen and scheduling an interview, at my next employer. If it ever took more than an hour from problem statement to solution, it was too long; although I heard of people taking 12+ hours. Some sent code that ran 12+ hours, and was wrong. (Our best ran in <1 ms, which no candidate matched.) Some candidates posted their solutions online, after being rejected, but every posted solution was really bad, so that was OK. Cheaters disqualified themselves.
In 2010 I solved the "bitvector" problem in the posted article, during two weeks of sharply heightened intelligence I had while adjusting to the new antidepressant. To this day I marvel at that code. But by then, there was noplace to send it. Google took down the puzzles a little later.
I sometimes wonder what Carl went on to do, after he left ITA.
I always tell them that undergraduates solve it quickly, but that never seems to help MSes.
There was a similar problem, back in the 80s, that made starting an Occam program on a network of transputers very, very slow, that was solved the same way.