Facebook’s Hacker Cup Called A Failure
allfacebook.com
allfacebook.com
Am I the only one who thinks Quora should only host questions and answers? I hope this isn't part of a growing trend to have what are essentially blog posts on Quora.
It's a shame because it really had the potential to be a fun competition if it had just been pulled off more smoothly. The questions were interesting and the competition format allowed for the use of pretty much any method of solving problems. The 6 minute limit was annoying at first but I came to see it as part of the competition. It creates an interesting tradeoff between development speed and correctness and allows people to solve the problem in any way they like whilst still enforcing efficient solutions. The bounds on all the variables in the problem are given so it is possible to test your solution on the worst case scenario before downloading the input.
What they need to do in future is have a better user interface that warns the user that they will only have six minutes to submit their answer after retrieving their input and then include a javascript countdown clock telling the user how much time they have remaining.
Facebook is more like undergrad. Something needs to be done, and people do it. Most of the time they don't read the literature on the subject, or consult experts about the "right way" to do it, they just sit down, write the code, and make things work. Sometimes the way they do it is naive, and a lot of time it may cause bugs or break as it goes into production. And when that happens, they fix their problems, replace bottlenecks with scalable components, and (in most cases) move on to the next thing.
http://www.quora.com/Which-is-better-to-work-for-Google-or-F...
The first post-qualification round was even worse (I was a competitor).
I won't go through all the details, but problems included mis-specified questions.
The second question - a 2 variable optimisation problem, set as a Starcraft game problem, in which shield generators cost X and warriors cost Y, and you had some rules about the cost/benefits of each - asked you to report the number of warriors to build, to maximise damage done.
Writing code to solve this, it quickly became clear that the sample answers provided were wrong.
Mid way through the competition, they changed the question to require you to output the number of shields to build - which, of course, made the sample output make sense. While they put up a post mid competition saying they suspected a bug in the question, they did not subsequently make any communication that they had changed the question, or confirm the error. No text was added to the question page notifying competitors it had changed. No confirmation of the change was officially made, afterwards. (You can verify this independent of my comment by reading the competition walls, or googling - the question was called 'power overwhelming').
Another issue - while they were technically correct - was where they said input to the first question would be whitespace separated, and give training examples showing it space separated, but then the test problems were Newline separated. While its debatable whether Newline is whitespace (I'd say it is, but it typically doesn't match \s) this change from the training examples to the test confused a lot of people - and its bad programming competition form, to trip people up in the input specification.
Finally, due to the slew of problems in the round, they closed the round early, without warning. This upset competitors who came to submit their solutions, to find that the round had already closed.
AFAIK, they still have not clarified whether competitors from that round that answered questions will progress, or will all have to redo it.
Communication - including on topics such as rules - has been sporadic, and either via e-mail, or through posts left on the wall - sometimes in comments on the competition wall, rather than in organised FAQs, or using the facebook notification system.
Answer upload has also been temperamental for certain browsers.
But posing a programming competition question to 1000s of competitors, where the actual question is mis-specified - that's a very big error.
Rescheduling the succeeding sub-rounds at the last minute also inconvenienced a lot of programmers who had taken time out of their schedules to do it.
Its a programming competition, and its just for fun - but I'd like to think they'd do a better job considering the volume of contestants, and the wide external audience.
[1] The Peg problem stated that there would be no ties, while the second example clearly had a tie. Again, they updated the tests, but only in the last 24 hours of the competition I believe, and with no notification to any of the competitors.
The next round was even worse, thankfully I spotted pretty quickly no one was solving the starcraft problem, an obvious problem to attract starcraft fans, and there are enough of those in the programming community. So gave that a miss and moved onto the others. Looks like most people got caught out on that problem, which was a real shame. They need to reset this round and start again to be fair, but I'm sure given how upset a lot of competitors were the server load won't be the same next time.
Rule 1: Read all the directions first. Rule 2: Jump up and down on your head Rule 3: Generate a proof which demonstrates that [insert random equation here] ... ... ... Rule 50: See rule #1 ... hand this paper in to complete the assignment within the first X minutes and you get an A. (or something to that extent)
While I gotta agree that the communication wasn't clear, part of me wants to be the grouchy teacher which says "the lesson is this: read the dang directions". This guy just forgot the 1st part which was: read the directions first!
The bigger problem in my book, however, was the issues with the submission system. I tried both Firefox and Chrome, and couldn't get the "download input" link to function properly (I tried, left clicking, "save as"-ing, opening in a new tab, etc.). Add the 6-minute time limit to this, and it was pretty bad. Six minutes to get the download to work, run your code on it, and then get back to the page, paste your output, and upload. Oh, and good luck eyeballing your output to make sure you pasted it correctly -- the window you paste into wasn't resize-able, and every line wrapped, so it looked like an utter mess.
Don't get me wrong, I sound rather negative in this post, but I love the idea, and look forward to the next attempt at it. I was just somewhat disappointed to see how the process went. And to be honest, I really wish they'd do away with the 6 minute time limit, and add a time limit for how long your code is allowed to take during execution. Though I suppose they have to restrict the languages then. Perhaps a "can you finish the problem within the 3 hour time limit" is sufficient for this competition then. As it is, knowing when to optimize, and when not to takes too long to figure out in the 3 hour time limit, so you end up taking a shot in the dark as to when you should be submitting your code.
Hopefully Facebook learns from these mistakes, and comes back with a competition that is just that much better. Also, if you're interested in the problem solving for this competition, check out the Puzzle Master. It's great fun as well.
I read all the FAQ stuff beforehand, so I thought I was prepared. I didn't know that there were additional rules/instructions when the contest started, because no one told me there were. So the first thing I did was read all three puzzles and download the test data for them. I actually solved the third puzzle and submitted my solution inside six minutes anyway, but I ran out of time on the first one (still submitted my solution after six minutes), and I didn't even bother with the second one because their sloppily worded FAQ didn't make it clear that getting through the qualifier round was competitive and that I needed to complete all three to have a chance at advancing. Their wording wasn't technically incorrect, but it gave the impression that they were just trying to filter out people who couldn't solve a simple problem.
I am pretty sure I'm the kind of person they were trying to capture with this competition, and, well, they failed to capture me.