a16z Programming Test
hackerrank.com
hackerrank.com
(Incidentally, PG designed the /jobs posts to be the one privilege that YC startups get on HN. Other than that, the site policy is to treat them the same way as everybody else.)
It's not uncommon for posts to be in a grey area between job ad and story. A programming test for an interview process doesn't seem all that grey, though, so I've been killing these. Sorry to be a hard-ass about this, but I don't want to be accused of enforcing the policy erratically.
Edit: I'll unkill the post (but otherwise leave it demoted) so the discussion can remain open.
Edit 2: The argument has been put to me that this is newsworthy because A16Z are doing it. Since that's corroborated by external sources (e.g. Twitter), and the post has gotten a lot of upvotes even while it was buried, that's reasonable—so we'll restore it as news rather than as an ad. Besides which, HN is one of the few places where a programming test counts as interesting.
In the meantime, I've been posting feedback comments most places that issues like this come up. You may need to look to the bottom of the threads to see them, because we usually mark them off-topic.
And you can always get answers to questions by emailing hn@ycombinator.com.
I'm quite interested in those tests, could there possibly be a way to see them?
In a tight labor market, people willing to do this are more desperate than others. So it's going to filter for desperate developers who passed a 3 hour basic skills test.
Old fashioned prior experience and education is probably still a better first filter. This would probably make more sense specifically for junior hires right out of school or with no experience.
Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other. This process is more fair and (IMO) allows someone without a college degree, or from outside the US, to compete on a level playing field.
> Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other.
So instead of hundreds or thousands of unstructured resumes in different formats, you have hundreds or thousands of completed programming tests. Even assuming that you automatically filter out submissions that fail testing, given that there are a lot of ways to solve a problem, you'd still be stuck evaluating potentially hundreds of correct submissions.
More importantly, a test doesn't seem to be aligned with your top priority:
> ...but the most important requirement is significant independent programming experience as demonstrated by your GitHub account, personal projects, academic publications, or startup success.
Are you assuming that all of the folks most accomplished in the real world are going to complete your programming test successfully?
Since you ultimately seem to be interested in folks who can build dashboards, I would think you'd be far better off asking candidates to build a dashboard prototype using sample data you provide. There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application.
I'm guessing the MVP of hackerrank was the web code editor + lint/findbugs/whatever for filtering. A lot of people have spent a lot of time writing open source projects that convert code to metrics, so making a reasonably prioritized list is probably fairly easy.
> There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application.
Agree!
I think this is a great critique of HackerRank as a business idea -- while they are new, it's just a hurdle that weeds the busier coders out of your hiring pool... if they become established there will be books on cramming for it, and will no longer separate the wheat from the chaff.
(ie https://web.archive.org/web/20130114163457/https://raganwald... meets http://en.wikipedia.org/wiki/Goodhart's_law)
Sorry - programming tests raise far more false negatives than false positives (and this from a person who considers himself a false negative in most tests)
Majority of the tasks IME:
- code reading & refactoring
- figuring out why two libraries aren't playing nice
- bug triangulation
- optimization (not the algorithmic type, but rather the fix-the-auto-generated-SQL-query / frontend asset packing kind)
- repetitive/simple/obvious bits of jQuery
Maybe there's some kind of online test that could exercise those skills, but if you ask me the main performance changers for modern web development are, they're: really knowing the stack (problem anticipation, what to use, how to use the various tools to quickly get search keywords), google fu, and most importantly having done it a couple hundred times (muscle memory).
- Communicating clearly and calmly with other people
Yes, binary trees are pretty much coded for you, and you would be a fool to roll your own hashing algorithm for production, but thats still not a reason to keep your head down.
The next decade is going to be fun. Trust me
I guess to reiterate my point: once you've done web development for a couple years, coding most websites is less about solving new problems and more about applying known solutions.
To reiterate how this point relates to the question of coding interviews: in as much as they don't test for lots of known solutions, and the ability to apply known solutions quickly, they aren't filtering for productive coders.
How does your response relate to the points at hand?
Think micro-services supplying REST APIs over HTTP - that's going to be same-y after a few years yes, but behind that is a world of business apps, integration challenges, complex algorithms and calculations - sorry I just see "web development" as "anything that has of will have an http access point" which I mentally count as "pretty much everything from here on in"
Programming tests do generally test for ability to apply known solutions quickly - it's about all they do do. You just have to know the known solutions (oh look calculate primes).
As for finding productive coders - well the amount of false negatives is large, and the false positive rate is probably higher (plenty of people who can waltz through these tests i find fairly poisonous to morale) so I am not a big fan - but if you need to make a first cut this is a good start
I think you missed my point. It's that the people submitting to a multi-hour exam just to submit a resume are more desperate / value their time less than those that can stand on their experience.
The fact that there a "quite a lot" of submissions only worsens the probability that the multi-hour effort will be wasted.
Just curious, why do you say that the normal process doesn't allow people outside the US to compete on a level playing field? Higher barrier to on-site interviews?
* By well-executed, I mean - relatively language agnostic, lacking in trick questions and meaningless trivia, and being open to multiple good answers. The intention is to not only check basic ability to solve simple problems, but the general approach and programming style of the individual interviewed.
Past experience is often vaguely stated in CVs and looks great on paper, but can be disingenous. It takes a thorough two-way discussion which is customised to each individual to reveal this, however. Some people have excellent and highly visible past achievements, but not everyone does. I wouldn't throw out a CV simply because the individual doesn't have any open source code to show me, nor do they have any awards from companies they have worked for (which is often more about self-marketing than actual ability).
I guess your mileage may vary based on the position that's in question and the companies involved, but I've seen some shocking discrepancies between CVs and actual programming ability. Even worse than the usual "fizz-buzz" statistics that get thrown around.
conversely my experience of education is that it trains you to solve education style problems - which are essentially absent from the real world - where being able to invent and implement a slightly broken solution in an hour with 30 seconds notice is much more valuable than being able to solve a problem you know about far in advance and are given loads of information and guidance to help in preparation for it.
on the other hand if i wanted someone to do research then a CS major might be a better choice. :)
Regarding CS majors, there's always the possibility of an individual being great at the theory part, but not having much programming experience. This should be rare though; during my own CS degree, we coded so much that I was frequently dreaming about code. If it's provided in the CV, I like to scan through the subjects list of the candidate - the ones who don't actually like programming seem to have an aversion to taking CS courses for elective subjects. As always though, YMMV, and it's a good idea to keep an open mind.
this kind of test is not valuable. the other tests i've done though - even though incredibly easy i think would be more useful for seeing how i approach problem solving in general...
If you're Andreesen Horowitz, you have access to a nearly-unlimited supply of capital. You also have incredible access to information and dealflow throughout the industry. In situations in which you are not capital constrained, the most efficient way to establish a workforce is to acquire it from somewhere else; that's why acqui-hires make so much sense, because you've basically had someone else do the hard (and often completely unpredictable) work of figuring out if people can work together in a successful manner (which, I should note, is different from determination of whether the thing they're working on is successful). If I were A16Z, I would arbitrage my information advantage to secure a high quality engineering team by acquiring a company that had smart people but a lame product; going through a traditional hiring process would be a waste of my time (and since everyone is throwing money at my firm, it would also be a waste of time that could be better spent on figuring out where to deploy my capital).
A16Z is rich enough to buy people, if it really wants people, for internal use (whether it is for their own company needs, or for some idea that they want to incubate inside of the firm). And even if they don't want to buy a whole set of people, they have enough wisdom and experience to know that you want to hire within a cohort of successful people rather than randoms (to see this in practice, take a look at Don Melton's post about Eazel alumni ending up at Apple on the front page right now). This tells me that their programming test has more to do with some sort of evaluation of this Hacker Rank service ("tell you what, we will run our own test on the service to see what happens!") or some other sort of analytical adventure.
Or, you know, maybe it is really a programming test after all. Maybe they are hoping to find some sort of diamond-in-the-rough engineers who are socially and career-wise disconnected from an existing network. Some sort of "black swan programmer" or something. But I highly doubt it.
So basically, if you go through this test, you're helping A16Z evaluate something; that something, however, probably isn't you.
Why acquihire someone who knows how much they can ask if you can pay close to nothing to that test website (it's a genetic service) and get a clueless buy competent nerd that will be glad to work alone on a project for peanuts?
That's why we're poor and they have "unlimited capital"
Building organizations is expensive, unpredictable and takes a long time. Buying organizations is more expensive in the short term, but also far more predictable and much faster. So if you have more money than time and need an organization to get something done, buying one is far more rational than "paying a clueless but competent nerd peanuts".
I'm not saying a16z is doing that here, btw. I'm just saying that you're wrong.
The team at Andreesen Horowitz, however, are not among these stupid rich people. If anything, they have disrupted the standard operating procedure among their peers by outsmarting them, and the nature of how they have outsmarted their peers is strongly connected to an appetite for attaching high valuations to small groups of really smart people. Their organizational M.O. has been the exact opposite of bargain-bin talent hunting, which is why this whole Hacker Rank thing comes off so weird to me.
I want to be clear that I don't think A16Z is up to anything evil. Just something sneaky (which is fun!). These are smart people, and for them to publicly engage with the hacker community in this manner is indicative of some sort of really smart hypothesis being tested.
I'll retract my statement if someone shows me real evidence that they've turned down a 500 (ok, perhaps 50 should be enough) million dollar job offer.
I don't have any evidence of any engineer being offered such exorbitant salaries , however this gets close.
http://www.businessinsider.com/a-google-programmer-blew-off-...
If you can find a reputed VC firm like A16z , whose looking to do what you propose (please post a link)
Ok, it's somewhat of a hyperbole. 50M for a single hire is extremely exorbitant (after all, we're not european soccer players), but consider being offered a salary 5 times your current pay.
I'm loyal to my employer as hell, even though he only pays me around 45k (euros, yes, i should get more ;) and I'd like to think I'm 'morally gifted', so I wouldn't consider quitting my current job when someone offers me double... 3 times plus, is a different story, I'm scratch my head at least twice.
If I were to guess, they're building an internal dashboard and DWH to crunch numbers on their investments (or perhaps potential investments). This test appears to focus on that skill set.
Yeah, they could probably buy a company, but that would be overkill. They're VCs, they're supposed to be selling companies at a premium, not paying premiums. ;)
You have to understand the way capital inside venture funds is usually allocated - it's not just one big checking account. A common way to describe the economics of a venture fund is "2 and 20" which means that the fund takes 2% management fees per year (across a 10 year fund lifetime) and takes 20% of the returns or upside.
So, in a hypothetical $300m fund with this arrangement, 80% of the money is reserved for making investments that provide returns. Every time an investment is made, a capital call takes place where the LPs are asked for the money.
The other 20% of the fund goes to management fees - 2% per year although it's not uncommon that these fees be non-linearly distributed, IE 3% in the early years and 1% in the out years when (presumably) new funds are also throwing off management fees.
So your $300m fund really generates $500k / month in management fees. (Probably paid annually or quarterly.) Enough for some very comfortable salaries, ample support staff, great office space and a travel budget but not enough to really start throwing dollars around millions at a time.
Now of course Andreessen has a reported ~$4B under management but I'd be surprised if they're really generating north of $6M in management fees per month. As fund size grows, management fees may come down (since it doesn't take 10x the effort to write checks 10x as large) or a fund as confident as Andreessen (and in as strong a position) might make a "1.5 and 30" deal where they take fewer dollars as management fees up front but get a larger percentage of the upside.
Note, I know absolutely nothing about what's really happening inside Andreessen, this is a purely hypothetical story against what is considered "typical" in the industry. (In the US.)
The real question is "What problems will these folks be solving?"
So, someone taking Statistics 101 is considered to know machine learning in the startup circus? (Not that this would really surprise me, given the tendency to appropriate terms that had a certain meaning for purposes that have only a passing similarity to the original meaning.)
But it's amazing how far you can get with some simple scatterplots, histograms and linear/logistic regressions.
a) Most of the time, more data beats better algorithms: http://anand.typepad.com/datawocky/2008/03/more-data-usual.h...
b) Even those seemingly simple things get complicated when you have a lot of data: http://komarix.org/ac/lr
In grading, the results of 6-8h of work entered in 2h would look more impressive than 3h of work in 3h, I suspect.
(Sometimes I hate that whenever someone gives me a situation, my first thought is "how can this be cheated or gamed", followed by "what controls can I put in place to prevent that, and and what costs"; it's basically automatic.)
That is they are not the 1% that the all hirers, but they are maybe top 20% and that is actually what most jobs require no matter the delusions of hiring managers.
Granted this particular job might be an exception but I doubt it.
If some employers want to implement recruiting processes that require developers to complete tasks that don't reflect real-world development, it makes it that much easier for other employers to differentiate themselves and attract competent developers who aren't interested or desperate enough to play these games.
Frankly, in today's market I feel sorry for any experienced developer who is going to complete a "programming test" in which referencing a website or book other than language documentation is forbidden. I reference websites and books all the time, and I don't know a single developer who doesn't.
What company do you run? Are you in the Bay Area? Can I buy you coffee? I have a feeling you could teach me things that are both true and that hackernews would downvote... And that there's a reasonable chance I could return the favor.
Background so you can filter me more easily: I'm a very good programmer (7 years experience coding on small teams made of up jQuery core, rails core, and MIT AI Lab alums. Several offers from YC companies. 4 papers & patents. Currently doing machine learning for the Scripps Institute of Oceanography). I'm also an avid reader of psychology (cognitive biases mostly), marketing (Joe Sugarman is amazing), and business books (I'm a raging Munger and Deming fan).
So I'm guessing if we did meet for coffee you at least wouldn't find it boring.
Email me? dls@lithp.org
0 - http://gigaom.com/2014/01/07/netflixs-cloud-architect-adrian...
Nope.
Similarly, I would infer that your suggestion is ignorant that they have interest in the companies post-funding (as cnaut suggests "A lot of VCs firms are trying to apply more data to their investment decisions and that requires internal tools to handle it all"... which I think would suggest that they would want to examine the results of said decisions as well (which goes beyond simply the submission process)).
Yes I am.
(1) It appears, really is quite clear, that they have high contempt for initial contacts from entrepreneurs via e-mail. Instead they want to believe that the 'good' entrepreneurs will have enough 'exposure', enough in 'networking', and enough in 'hustle', e.g., on LinkedIn, GetHub, etc., to come to the attention of A16Z by means other than an e-mail message "over the transom". E.g., they want an entrepreneur to get introductions from bankers, lawyers, accountants, or CEO's of companies in the A16Z portfolio. So, A16Z (A) wants others to 'filter' their entrepreneurs for them and/or (B) believes that lawyers, etc. are good sources of good recommendations. Nonsense.
(2) A16Z will just refuse to consider any very technical support for a project. E.g., if a project has some powerful, valuable 'secret sauce', intellectual property, and high technological barrier to entry, A16Z can easily just dismiss the technical issues, the project, and the entrepreneur. A16Z wants to believe that they have 'deep domain knowledge' where they invest and, thus, wants to believe that anything new that A16Z has not heard of must be nonsense. If some supporting technical material is not easy reading for them, then they believe that they are justified in just regarding it as nonsense.
But, looking at the partners of A16Z, it's tough to find one with enough technical qualifications and/or accomplishments to (A) get a tenure track position in the engineering school of a leading research university, (B) be an editor of a leading technical journal of original research, (C) publish a peer-reviewed paper of original research in a technical field in a good or better journal of original research, or (D) get an NSF grant for original research.
Net, A16Z apparently is not competent to review or even to direct a competent review of technical material in information technology.
What the A16Z financial performance really is is likely not easy to discover.
You seem to suggest that the project for which the OP is some recruiting is for an internal information system to do causal modeling and make valuable predictions for venture investing. It would appear that they want to do cross tabulation and curve fitting; sorry, there is nearly no chance those techniques will work. Why? In simple terms, they are looking for new, exceptional cases, and looking at 'averages' from the past, even of successes in the past, is just intellectual self-abuse.
If they are making money, then, fine. But from all I can see, in information technology their competence is way below a level that would be useful for actual technical work or doing a technical review of such work.
I've been around technical work in US national security, US business and start ups, including one of the most successful venture funded companies ever, and high end academic research, hold a Ph.D. in engineering from one of the world's best research universities, have published peer-reviewed original research in computer science, applied mathematics, and artificial intelligence, and in comparison the technical qualifications of A16Z look like a bad joke.
Yes, most good quantitative analysis needs some computing and programming. So, in their recruiting A16Z seems to believe that the computing, which is necessary, is also sufficient. It's not. By an analogy, A16Z is looking for nurse practitioners instead of MD physicians. Both are necessary, but the nurse practitioners are not sufficient. There is a difference!
But, why help A16Z? They have money enough now!
- First, what is your sample size? Do you have any idea how many meetings they give come through email? Just because they may have rejected you (no idea), doesn't mean they reject everyone. Second, I would suggest that a cold email is probably the worst way to approach ANY VC.
"A16Z wants to believe that they have 'deep domain knowledge' where they invest and, thus, wants to believe that anything new that A16Z has not heard of must be nonsense. If some supporting technical material is not easy reading for them, then they believe that they are justified in just regarding it as nonsense."
- I would disagree. That being said I think you're misunderstanding the point of the VC business. It is NOT to create / promote new technologies only. It is to promote businesses and segments that they believe will be profitable (in general).
"But, looking at the partners of A16Z, it's tough to find one with enough technical qualifications and/or accomplishments to (A) get a tenure track position in the engineering school of a leading research university, (B) be an editor of a leading technical journal of original research, (C) publish a peer-reviewed paper of original research in a technical field in a good or better journal of original research, or (D) get an NSF grant for original research."
- Ok. But again you're measuring the success of a VC on technical acumen. What are technologies generally used for? To solve problems. In business domains. You want to go through that partner list again and tell me they don't have experience in various business domains?
"Net, A16Z apparently is not competent to review or even to direct a competent review of technical material in information technology."
- They don't make a claim to. Next?
"What the A16Z financial performance really is is likely not easy to discover."
- Let's see. Google "a16z performance returns". First article: http://finance.fortune.cnn.com/2012/07/23/nice-ira-andreesse... . I must be a genius. Yes, this is but the first data point; however, it's a start.....and based on other more recent data points (oculus, etc.), it may be suggested they're doing well. It's nice that you want to brush how most VCs are judged as successes or not (by the performance of their portfolio) as secondary in comparison to what you want to judge them by.
"You seem to suggest that the project for which the OP is some recruiting is for an internal information system to do causal modeling and make valuable predictions for venture investing."
- I'm sorry you inferred that. Allow me to be perfectly clear. I have no idea what they are recruiting for. I could imagine a lot of projects....the above is something that is certainly not out of bounds
"It would appear that they want to do cross tabulation and curve fitting; sorry, there is nearly no chance those techniques will work. Why? In simple terms, they are looking for new, exceptional cases, and looking at 'averages' from the past, even of successes in the past, is just intellectual self-abuse."
- It may appear like that to you; however, I would suggest it's probably much more complex. They could be performing (for example) some clustering techniques (k-means, k-medians, what not) across the investments. They could be performing a variety of analytics on the talent development portion to gain a better understanding of expectations of the companies and candidates. They could be doing a lot of things...
"If they are making money, then, fine."
- I'm pretty sure that's not the case with you. I'm pretty sure that even if it was shown they're beating even 80% of other VCs you would still be hyper critical of their technical evaluation skills.
"I've been around technical work in US national security, US business and start ups, including one of the most successful venture funded companies ever, and high end academic research, hold a Ph.D. in engineering from one of the world's best research universities, have published peer-reviewed original research in computer science, applied mathematics, and artificial intelligence"
- Do you want a cookie?
"and in comparison the technical qualifications of A16Z look like a bad joke."
- Yet their business and venture capital qualifications appear to be quite good.
"By an analogy, A16Z is looking for nurse practitioners instead of MD physicians. Both are necessary, but the nurse practitioners are not sufficient. There is a difference!"
- They're not looking for MD physicians as the portfolio companies have the technical expertise.
Sorry they are having trouble handling e-mail. It's an old technology and relatively easy to work with. E-mail is just simple text, much like HN posts. Such simple text works great at HN. Too bad the 'high tech' A16Z can't work with simple text.
With all their efforts to find good investment opportunities, they might just look at their incoming e-mail and then read it. But your statement seems to indicate that they want to ignore e-mail, and that does appear to be the case. Ignoring e-mail is bizarre, but they are free to do so.
For the rest of your post, you miss the big, huge point: As is commonly asserted, one of the better parts of a new project is new technology that is powerful and valuable, intellectual property, 'secret sauce', with a high technological barrier to entry, for solving a big, important problem. Such technology, at least in my interest, is definitely just in the service of making money, the green kind.
If a project claims to be able to solve a huge problem so far unsolved, that many others have tried unsuccessfully to solve, then a crucial, early question for interest in the project is what the heck is the crucial 'secret sauce' that enables the solution, that lets this project do what prior projects could not?
Again, of course, the purpose of and interest in the secret sauce is making money.
I can 100% absolutely, positively guarantee you that one can write a very carefully written e-mail with a project to solve what is clearly a huge problem with overwhelming and obvious evidence that a company solving this problem will be worth $x billion for x high enough to be historic, and outline some of the crucial secret sauce and have A16Z totally ignore the contact. Call them on the phone and tell them about the e-mail, and they will refuse to dig it out of their inbox and look at it. They will say, "send it again", and someone might say, "I don't think that we can do business along these lines.". "Send it again" is just a manipulation to insult entrepreneurs by a firm that does not value contacts from entrepreneurs. If I get rich, A16Z won't participate.
The importance of their abilities to evaluate new, powerful, valuable technology is just that such evaluation is just crucial for any project where one of the main features is such technology. As I made clear, thankfully for US national security, the US DoD has long been excellent at evaluating new technology. If the DoD sees a significant problem and someone comes up with some new, powerful technology for the first or a much better solution, then the DoD will listen and, really, in my experience, for a good case, proceed. My experience is that apparently mostly A16Z won't; they won't even look at their e-mail or listen on the phone. Heck, just finding their phone number takes some effort.
Biomedical VCs will evaluate such technology. Information technology (IT) VCs won't. Why? The IT VCs just don't want to care about such technology. Instead they want to make money other ways. As posted by Fred Wilson at his AVC.com some months ago, on average the IT VCs are doing poorly. Still they insist on not even evaluating new technology. Their feet are locked solidly in concrete with their eyes and ears closed, refusing to pay attention to e-mail and phone calls.
Really the IT VCs believe that all there is to the 'technology' in IT is little more than routine software development. On average, this attitude fits with the past. But so far, e.g., from A16Z there are only about 15 projects a year that deserve a Series A. And it's easy to see that there are only a few Googles or Facebooks each decade. So, looking at the average projects in the past is a hopelessly poor way to find the 15 Series A projects for this year or the next Google this decade. Still, A16Z won't read their e-mail to find such projects.
The situation is fairly clear: The IT VCs want to see traction, up and to the right. Then they want to invest for the big build out and the rapid growth on the way to going public. Sometimes they make money doing this; on average essentially they don't.
For your
> - They're not looking for MD physicians as the portfolio companies have the technical expertise.
No. My remark was in the context of the project of their OP and recruiting. For the goals of that project mentioned in this thread, my analogy with an MD is correct -- they will need one. But they are not recruiting for one. Instead they are recruiting for nurse practitioners. E.g., you mentioned cluster analysis. So, as I explained in terms of the SR-71, could not evaluate that project by cluster analysis of the then history of military aviation and, instead, need to look at the engineering details Kelly Johnson was proposing. Similarly, cluster analysis stands to be nearly hopeless for predicting the future of IT projects and, instead, must look at the details of the projects. It does appear that the project of the OP missed this point.
> - Do you want a cookie?
It's really tough to communicate with someone who works not to understand. Then even something simple won't come across. The point was, in really simple, baby talk terms, (1) the goal if IT projects is to make money. (2) One of the best possible parts of a new IT project is new, powerful, valuable, technology to provide the first good or a much better solution with a high technological barrier to entry to a big problem where such a solution will be very valuable. (3) To evaluate such a project early on, it is just crucial to evaluate the technology. (4) My background shows that I am competent both to create such technology and to evaluate it, and, more generally understand the potential of such technology in projects in business.
From that background, no, I don't want a cookie. Instead it would be good if A16Z would read their e-mail and evaluate projects including any new, powerful, valuable technology. It's not about cookies. It's about evaluating projects.
I wrote
"If they are making money, then, fine."
and you responded
> - I'm pretty sure that's not the case with you. I'm pretty sure that even if it was shown they're beating even 80% of other VCs you would still be hyper critical of their technical evaluation skills.
Sure. However much money they are making, they shouldn't pass up making much more. The US DoD doesn't do this. Neither does the NIH or the US pharmaceutical industry. People in US national security, biotechnology, and research in mathematics, statistics, and most areas of engineering would say that refusing to evaluate technology must be playing with mice and ignoring elephants.
Besides, US VC ROI is so bad that being better than 80% of the US VC firms might well still mean just losing money for the LPs.
> It's nice that you want to brush how most VCs are judged as successes or not (by the performance of their portfolio) as secondary in comparison to what you want to judge them by.
No, I judge them by making money. Their business they have chosen is IT VC. Then if they refuse to evaluate technology, then they are at high risk of missing out on the next big win of the next decade. Why is this clear? Not really because 'secret sauce' played such a big role in most or even any of the big wins in IT in the past 20 years but because of the overwhelming power of 'secret sauce' demonstrated in US national security and the biomedical industry. The NSF funds research in the mathematical sciences and more mathematical ares of engineering for a very good reason; but Silicon Valley (SV) believes that no business value can come from that work. SV is wrong. They may be very badly wrong, e.g., miss a few new Googles.
I wrote
"Net, A16Z apparently is not competent to review or even to direct a competent review of technical material in information technology."
and you responded
> - They don't make a claim to. Next?
Let's see: At their Web site can see
> At a16z, we bet on entrepreneurs who use software to go after the big problems.
Hmm. Sorry, guys, considering just routine software is a huge handicap in going "after the big problems".
> We believe this is an incredibly exciting time to be a technology investor.
Okay, here they say "technology" and not just routine software. But do they want to evaluate technology? Apparently not. Heck, they won't even read their e-mail.
> smartphone users are expected to grow from 1.5 billion today to five billion in the coming years.
There they go thinking of sectors again. Maybe with all those new users there will be new problems to solve and valuable companies to solve them. Otherwise they are talking a "rising tide raises all the boats" growth of a factor of only a little over 3 in 5 years, and that's not very impressive. Broadly, super tough to get venture returns from growth of a sector and, instead, need to get such returns from growth of individual projects, maybe in a rapidly growing sector, maybe not. For such projects, 'secret sauce' technology can be one of the best advantages.
> These entrepreneurs care enough about all aspects of their product/service that they want (or need) to innovate in all the areas that touch it.
So, now they also want to "innovate". That's what I'm talking about where you thought it was just a cookie. But such innovation needs evaluation. Of course, if A16Z just looks at the traction, they they can talk about software, technology, and innovation, refuse to evaluate anything technical, and just count monthly uniques, page views, ComScore data, etc. But A16Z says that they are in "multi-stage" investing, and waiting for traction now risks being quite late, too late.
Too late? Just do a little of the arithmetic all of use here at HN can easily do: Pick an Internet connection with, say, 25 Mbps upload bandwidth. Pick a server with, say, an 8 core processor at 4.0 GHz and 64 bit addressing with 32 GB of ECC main memory and an armload of 4 GB disk drives. We're talking $1500 in parts. Send Web pages for 400,000 bits per page. Send 5 simple ads per page. Get enough users to half fill that upload bandwidth 24 x 7. Have the software fast enough and scalable enough that just one or a few such servers can handle the load. Assume $2 per 1000 ads displayed. Okay, let's multiply it out:
So send
25 * 106 / ( 2 * 400,000 ) = 31.250
pages a second.
Then revenue of
31.250 * 2 * 5 * 3600 * 24 * 30 / 1000 = 810,000
dollars a month.
Now, just why does such a project want to accept seed or Series A funding?
Has any project successfully thought this way? Sure: Early on Plenty of Fish was just one guy, two old Dell servers, ads just from Google, and $10 million a year in revenue.
> If we are right that software is in fact eating the world,
There is considerable question if just routine software will yield venture returns. One reason is the low barrier to entry. E.g., China had no trouble doing something like Google. Neither did Microsoft. A16Z seems to accept low technological barriers to entry; that has to be a big mistake, even if they are making money. Heck, my local pizza guy is making money.
I'm failing to see why an entrepreneur with a good project should waste time with A16Z who hide their phone number, refuse to read their e-mail, and won't and even can't evaluate crucial secret sauce technology.
- I would suggest they're not having trouble handling email...their filtering is working just fine. You would appear to have trouble handling rejection (at least I would infer that)
"But your statement seems to indicate that they want to ignore e-mail, and that does appear to be the case. Ignoring e-mail is bizarre, but they are free to do so."
- Allow me to be clear. They are not ignoring e-mail. They are selectively choosing what they will spend their time on. They have (apparently) chosen not to spend time on your idea. Perhaps in the future you should make your emails more clear, you should be more persuasive, and your idea more compelling.
"For the rest of your post, you miss the big, huge point"
- Oh I get it. I just have quite a bit of contradictory evidence in how they work. Let me paint a picture for you. Perhaps you've heard of Prof. Cheriton of Stanford. He's written a publication or two and invested in a successful startup or two. He invested in a startup called AsterData. So did A16Z. Do you see how that works? They don't necessarily need the deep technical expertise within their team. They can rely on others to do that. Similarly, you might recall me mentioning that cold email is not the best course. If the technical idea you're suggesting is so revolutionary, you should have little to no problem having a well-respected professor get you an introduction.
"I can 100% absolutely, positively guarantee you that one can write a very carefully written e-mail with a project to solve what is clearly a huge problem with overwhelming and obvious evidence that a company solving this problem will be worth $x billion for x high enough to be historic, and outline some of the crucial secret sauce and have A16Z totally ignore the contact."
- Actually you can't. That's hyperbole again. Showing a problem and presenting a solution w/o evidence that you'll be able to execute it in a compelling manner would appear to be more of the problem for you.
"If I get rich, A16Z won't participate."
- I sincerely hope that works out for you.
"If the DoD sees a significant problem and someone comes up with some new, powerful technology for the first or a much better solution, then the DoD will listen and, really, in my experience, for a good case, proceed."
- So if some random PhD just emailed Lemnios, he would pick up the phone and call them? Doubtful..sorry. (and yes...i did my time with DISA as well)
"My experience is that apparently mostly A16Z won't"
- And mine contradicts yours.
"on average the IT VCs are doing poorly" - Fantastic, we're not discussing "on average"...we're discussing A16Z.
"Still they insist on not even evaluating new technology." - Incorrect. What about Oculus for example? Or do I need to provide more examples?
"Their feet are locked solidly in concrete with their eyes and ears closed" - You might be looking in the mirror.
> - Do you want a cookie? "It's really tough to communicate with someone who works not to understand."
- Boy, do I ever agree.
"It's not about cookies. It's about evaluating projects."
- Perhaps they evaluated and weren't impressed. Or is that impossible?
"US VC ROI is so bad that being better than 80% of the US VC firms might well still mean just losing money for the LPs."
- Fantastic. Again we're not talking about the broader VC industry.
"No, I judge them by making money. ... Then if they refuse to evaluate technology ... ...But do they want to evaluate technology? Apparently not. Heck, they won't even read their e-mail."
- You seem to keep missing the obvious point of perhaps they evaluated the technology you seem to be so bunched up about...and didn't care for it.
No, I'm just reporting some strong evidence that they just don't read their e-mail. Or, to be more clear, apparently essentially all information technology (IT) VCs will just ignore an e-mail message saying (generic version; real version with specifics):
(1) Here is the problem we are attacking. So far it is easy to see that there is no good solution. Fairly clearly a good solution will be of high interest to over 80% of all the Internet users, work station down to smart phone, in the world.
(2) The main reason the problem has remained unsolved is that knowing how to solve the problem is a challenge. The team has a new, unique, powerful, valuable, rock solid solution that has a high technological barrier to entry.
(3) Monetization? With a few billion unique users a month, for a first cut, just run ads. For more, there is an aspect of the project that permits some especially effective ad targeting with especially good protection of user privacy.
(4) The crucial core software implementing the technology is production quality and scalable. The rest is just an especially simple front-end Web site, nearly done.
(5) Team qualifications in business, research, and software are high.
Then with 1-5, they will just ignore the contact. No response. No questions. Nothing. They just ignore it. From the claims there, we're talking $100+ billion exit value; they just ignore it. No response; just toss it into the bit bucket.
Look, we're talking a billion or more unique visitors a month and an exit value north of $100 billion, and they have no response at all. Strong evidence that they just ignore their e-mail. Leave a $20 on a sidewalk; a guy walks by; the $20 is still there; conclusion, he didn't see the $20. Or, the VC firm just didn't see the e-mail.
One reason is, US IT VCs just have never seen any such thing, that is, an IT start up where what is crucial is the 'secret sauce' technology to solve the big problem. Instead, they pursue only other ways of making money and where the technology is essentially just routine software. US IT VCs and A16Z just will not give any value to (that is, won't spend time to investigate) results of research in IT -- the DoD is awash in valuable results of such research; biotech VC will take research seriously; but US IT VCs haven't seen any big business success from such research, don't really understand research or how to evaluate it, want to pursue other ways of making money, and want just to ignore anything based on research. They just want to f'get about research. That is the obvious explanation.
Since the US IT VCs are just determined to ignore research, the flip side of that is an advantage for any project with some good research.
Net, they don't read their e-mail and won't evaluate technology.
> - I would suggest they're not having trouble handling email...their filtering is working just fine. You would appear to have trouble handling rejection (at least I would infer that)
No, the situation is just as I said: The evidence is overwhelmingly strong that they just refuse to read e-mail from someone they don't know. That's their choice.
> They don't necessarily need the deep technical expertise within their team.
Yes they do or they are at risk of missing out on possibly some of the best projects. As I made clear, for the past 10-20 years, information technology (IT) VCs have not seen big successes based on anything at the level of publishable technical research. So, they just f'get about thinking about or evaluating such work. Again, the track record and promise of such research is from DoD, NSF, NIH, etc., not IT entrepreneurs over the past 10-20 years.
Research based projects will likely remain only a tiny fraction of what US IT VCs get in their inbox, but such projects are a good bet for a few $100+ exits over the next decade. Considering how few such successes there are, ignoring such projects, especially with the example of DoD, is foolish.
For Cheriton, they were not reading just some e-mail or investing in the technology; instead, they already knew the professor and were investing in him. But professors are not nearly the only people who can do research.
> - Allow me to be clear. They are not ignoring e-mail.
No, flatly they are ignoring e-mail. Write them something short, clear, and astounding, and they just ignore it, won't read it, won't respond at all. Maybe they want an introduction. Maybe they want only other means of selecting entrepreneurs. Maybe they just are having fun pissing off entrepreneurs. Maybe they are busy enough with what they are doing now. Maybe whatever, but they ignore e-mail. Sorry 'bout that, but they do.
> - You seem to keep missing the obvious point of perhaps they evaluated the technology you seem to be so bunched up about...and didn't care for it.
Nonsense. For the technology in question, there is likely and apparently only one person in the world who knows it. Send A16Z a high level view, and they ignore the communications. They make no attempt to get the details of the technology and evaluate it. It's simple: They just will not get involved in evaluating new technology.
> - Perhaps they evaluated and weren't impressed. Or is that impossible?
It is impossible. They can't have even as much as a weak little hollow hint of a tiny clue about anything about the crucial internals of the technology. They received only a business view of the power and value of the technology. For how it works, which is crucial for evaluating it, they don't know. To know they would have to do some due diligence. There is little hope that they could competently direct the due diligence.
> - Fantastic. Again we're not talking about the broader VC industry.
Right. We're talking about A16Z. My point was in response to your figure of 80%: Likely a VC firm with ROI just better than 80% of the VC industry is still losing money for its LPs. In this case, even if A16Z has better ROI than 80% of VC firms, they might still be losing money. However, since they just raised $1.5 billion, I suspect that it appears that they are making money; since a lot of the old investments likely have yet to exit, it might be tough to know their ROI so far, but no doubt they had to show something good about ROI to raise $1.5 billion.
> Incorrect. What about Oculus for example?
I don't know that case well, but my superficial understanding is that the product was in production and selling and all a venture firm had to do was to evaluate the sales figures and the product itself without much concern for any crucial internals of the technology. Like I said, the US DoD, CIA, etc. funded the SR-71 when nearly it was was engineering on paper, and VCs would want to wait until the plane was flying successfully and then maybe pay for some of the jet fuel. US IT VCs just will not evaluate technical work just on paper; they won't do it. Maybe this situation is enforced by the LPs, but that is the situation. The story used to be that VCs could fund a project described on the back of a napkin, but now they just will not fund original technical work just on paper or even when the corresponding software is production quality and running. If some such project gets funded, then the reason was not just the technology and software but the reputation of the founder or some such. NSF, NIH, DARPA, Ph.D. committees, and peer-reviewed journals of original research will evaluate new work just on paper, but US IT VCs just will not. And, e-mail with any such connection with such research goes into the bit bucket, from their "filter" if they look at the e-mail at all.
We are in agreement: They just do not want to look at e-mail.
So be it.
The project in question, if it starts to work at all, will quickly generate revenue of $10,000, $40,000, ..., $800,000 a month starting with just a $1500 server and, for $1 million a month, using just a few such servers in a spare bedroom with some A/C in a window, a consumer Internet connection (that permits commercial usage), and a 200 A circuit breaker.
So, why jerk chains of VCs? For the project in question, it has long been the case that there has been no very good reason, although maybe just as 'reserve fuel' in case of some disaster. The main reason for my posts here are just to let the HN community know that A16Z and essentially all of US IT VC just will not take seriously descriptions of technical work in e-mail without an introduction, knowing the person, etc. For anyone with a background in doing projects inside a business, for US national security, or for academic research, this situation is just bizarre and has cost the founders of the project a lot of time although only some time ago. But the project moved forward on its own funding, and revenue with good free cash flow, plenty enough to grow capacity if usage does grow, seems quite near. So, for this project, to heck with VCs. But the time wasted in the past on VCs still hurts. And the HN audience should know some of the truth.
The US DoD, DARPA, NSF, and NIH have long shown the world "how it's done", in total for 70+ years.
The simplest arithmetic shows that US information technology (IT) venture capital (VC) is not a very good business compared with consumer Internet: That is, a US IT VC firm might get up to assets under management of $5 billion or so, but the founders of Google, Facebook, and more have more than that in their personal net worth.
One problem with VC is, even if they decide to take their e-mail seriously, really the best they can do is just from what arrives in their e-mail, and far from VC there are some serious obstacles here: First, a standard assumption is that, since computers are to be used in the work (they likely do remain the great largely unexploited opportunity), the assumption is that computer science is the best academic background. Well, that assumption is false; if want to be quite serious about the work, then that assumption is badly false. Second, for really good results, it will be necessary for some good people with good academic backgrounds in crucial fields and topics outside of computer science to pursue entrepreneurship, and so far the appropriate US academic culture discourages this.
For this assumption, the US DoD knows that it is largely false and, thus, concentrates on pure and applied mathematics, mathematical and applied physics, and some relatively mathematical topics in electronic and other parts of engineering. The DoD makes very heavy use of computing but not so much of computer science.
So, for good results in VC, it will be necessary to catch up with the DoD and get good people with the 'right stuff' from academic fields other than computer science, and that catch up effort will be difficult.
Really, for good projects, the situation is much better on the entrepreneur's side of the table because there one can cook up terrific stuff instead of just waiting for it to arrive via e-mail.
My point that VCs and A16Z ignore their e-mail really was well supported by your statement that e-mail is about the worst way to contact a VC partner. Okay, the VC partners don't want to take e-mail seriously. Bizarre, but true, and so be it.
For the project in question, once it gets enough 'traction' to be of interest to a VC, the revenue will be way past when the project would accept equity funding. E.g., one VC responded that they wanted to see 100,000 unique visitors a month before investing. Okay, but for the project in question that would be about $40,000 a month in revenue from a server with $1500 in parts and an Internet connection costing about $100 a month. With the $40,000 a month, VC equity funding, a C-corp with VCs on the Board, etc. would be about as welcome as a skunk at a garden party.
One big reason: For the project in question with its technology the VCs can't evaluate, there will be more such work to do, less important than the work done already but, still, very much worth doing. But, of course, the Board would have to approve the organization and budgets for such work. Tilt! A Board of VCs would not be able to do such work because they would not be able to understand the project they were being asked to approve. So, such a Board would a hole in the bottom of the boat of the company.
The current A16Z Web site mentions that the number of VC firms has been shrinking. Since they refused to read their e-mail, easy to believe.
Darwin has a lot of work to do and, thus, is a very busy guy; still Sand Hill Road and Winter Street are due for a visit from Charles just any day now. Likely what will be left are firms willing to read e-mail and evaluate projects with new, advanced, powerful, valuable technology that is the crucial core of a valuable business. In the meanwhile, entrepreneurs should concentrate on some good research for some good projects that can be brought to market and significant revenue with, say, a server from $1500 in parts and an ordinary Internet connection. The research is the crucial part.
Sure, VCs won't invest in projects based on research, not even if the research is already in production quality code. But think of the flip side: For such a project, VCs won't invest in any competition, either!
That's definitely the plan except for the part about "mere mortals".
Mortals suffice, but not "mere mortals": Instead need some help, or, say, need to "stand on the shoulders of giants". The giants in question are not in computer science -- "a field with much that is new and good however the new is not good and the good is not new", too close to being fully true.
Without standing on the shoulders of the right giants, your "mere mortals" is fully correct -- can't do it. That is, one guy has, in practice, no chance of reinventing all that stuff needed from the giants. Indeed, it's tough enough just to get through the exercises in one of the better books needed. More than one guy? For the original work needed, that is nearly always done by just one guy; two are less good.
E.g., how to use pattern recognition in the industry of leading edge military aviation to evaluate what Kelly Johnson said about building an airplane that would go at Mach 3+ and 80,000+ feet for 2000+ miles without refueling? Can't do it. Instead have to look at the mathematics, engineering, etc., especially a special engine just developed at a special Pratt and Whitney site in Florida.
Information technology venture capital needs to find projects that have powerful, valuable new work and are exceptional. There is no way to evaluate such projects by simple, empirical 'pattern matching' from the past of business and venture funded projects.
Venture capital looks at sectors, the demeanor of the founders, maybe their socks, the jut of their jaw, etc. because that is all they know to look at. One could count on two hands all the information technology venture partners in the US able to do a competent review of projects submitted to the NSF. E.g., the last paper I published in computer science is for a quite practical problem in practical computing, but I doubt that anyone at A16Z could read that paper or even direct a competent review of it.
Thankfully for US national security, how to review projects in technology is very well known and done very well everyday, for 70+ years, by NSF, NIH, DARPA, etc. For Silicon Valley for information technology (the situation for biomedical technology is quite different), there's essentially no chance of competent technical review, not at A16Z, KPCB, Sequoia, Benchmark, etc.
For an analogy, for the SR-71, Lockheed could build several, have some fly successfully over Russia and get some fantastic pictures, and then venture capital might invest to buy some of the jet fuel. Instead, the USAF, CIA, etc. did some really good work reviewing the really excellent work of Lockheed, all from what Lockheed submitted just on paper.
Net, Silicon Valley venture capital just absolutely, positively, flatly doesn't have even as much as a weak little hollow hint of a tiny clue about how to do or evaluate original, exceptional, powerful, valuable information technology projects. Sorry 'bout that.
- Your use of hyperbole is amazing. The original point of venture capital was to "aid in the development of new or existing businesses into companies of stature and importance". Venture capital arose after world war 2 when banks wouldn't bankroll some of the businesses and new ideas that were coming out. Since then it could be argued that VCs direction has changed to investing the 3rd party pooled money into businesses that are employing some novel technology or processes. VCs typically have employees who are familiar with industries, business, and / or technologies to help identify candidate investments as well as to help said investments to flourish. Finally to brush the successes of venture capital over the years in IT projects is just disingenuous. VCs had quite a bit to do with DEC, Fairchild, the whole damned semi conductor business, and the expansion of the computer industry. Was a lot of it on the backs of other projects and companies? Of course. DARPA, Xerox, etc. played HUGE roles in the advancement of technology.
The information technology (IT) VCs are essentially hopeless at the crucial work of evaluating the "novel technology". The US DoD, NSF, NIH, and DARPA are quite good at such evaluations.
> VCs typically have employees who are familiar with industries, business, and / or technologies to help identify candidate investments as well as to help said investments to flourish.
For good exploitation of technology, what you are describing is not nearly good enough. The IT VCs need very much to up their game to whole new levels. They need, say, to be able to do or at least direct evaluations of anything that arrives in their e-mail in IT that might make a big pot of money, and they can't and won't do that.
Or they're simply handling recruitment for a stealth portfolio startup.
Why? The VC firms never invest in an "industry", i.e., there are no such 'index funds'.
Instead, VC firms invest in individual companies, and there what is just crucial is what the company does. If the company looks good, then mostly to heck with the darned industry.
> Or they're simply handling recruitment for a stealth portfolio startup.
In this case, poor startup: A16Z is doing a "piss poor" job 'helping' the startup!
Mattermark has gone some way to provide proof that data can make a difference, at least at highlighting opportunities that might not have been visible before. I wondered whether this would result in herding investors towards some companies, which means differentation and opportunity comes from spotting things the others miss... finding the outliers that Mattermark miss.
Interesting that the test is "make a dashboard", well into the standard skillset for most analyst programmers who work for financial institutions (I built a lot of this stuff for the finance sector in London).
The overwhelmingly largest share of returns accrue to capital. It's a good idea to resist being cast in the role of simply "programmer" over time, for this reason.
Also, who even worries about spam any more? I haven't seen actual spam in my inbox in years.
Spam remains an issue if for whatever reason you're still using rulesets/blacklists/greylisting and the like, instead of Bayesian filtering.
The constraint that gets me is the honor code - I cannot use an IDE with code completion?! but that leaves nano or notepad ...
"You may use reference language documentation or use an IDE that has code completion features. "
what's the first thing your teachers said ? Read the instructions carefully :-)
Hopefully they send some feedback... I'm not looking for a job, just wanted to know how I would do if I were:)
Besides, I love programming challenges.
Something to be aware of if you are considering doing this... it looks like it really is for employment and not a challenge.
I guess one of the first question "Are you ready to go to work in Menlo Park CA" (No) should have probably clued me in:)
Oh well... they will "be in contact". Right.
So, unless something changes, which I don't predict, it was a waste of time I wouldn't recommend doing the test nor applying to work at a company like this.
A more realistic programming test would be a single, complicated problem with an "end of day" time restraint.
"the most important requirement is significant independent programming experience as demonstrated by your GitHub account"
a great way to find newbie hipsters and not experienced and seasoned devs. or at least limiting yourself to one community
A BS/MS/PhD in Computer Science or the equivalent is nice
to have, but the most important requirement is significant
independent programming experience as demonstrated by your
GitHub account, personal projects, academic publications,
or startup success. Your accomplishments are much more
important than your paper credentials.Issues:
(1) Engineering School. Uh, guys, here in the US we have universities with schools of engineering -- MIT, Johns Hopkins, CMU, Georgia Tech, etc. Running a good engineering school is a lot of hard work. But this A16Z project has A16Z being their own engineering school, that is, specifying 'course material', testing for it, and deciding on pass/fail. Somehow it seems a bit arrogant for A16Z to believe that they can do the work of an engineering school.
A16Z gets really off the track with their,
> A BS/MS/PhD in Computer Science or the equivalent is nice to have, but the most important requirement is significant independent programming experience as demonstrated by your GitHub account, personal projects, academic publications, or startup success.
Wow! "PhD in Computer Science" ... "is nice to have, but"...! Gee, guys, just shut down MIT, Johns Hopkins, CMU, Georgia Tech, etc. because now A16Z is here with "the right stuff"!
Next, for their "as demonstrated by ... academic publications", sorry, A16Z, "academic publications" are not at all about "programming experience". Instead, the usual criteria are "new, correct, and significant" where "programming" really does not count more than, say, just the word processing for the paper.
(2) Machine Learning. Good grief! That "machine learning" is about "scatterplots & histograms, logistic/linear regression, R/Matlab" fills a "much needed gap in the literature" and "would be illuminating if ignited". Their definition is for a 'statistical data assistant' mostly just for 'descriptive statistics'. Ever hear of J. Tukey? Ah, I'll be nice to you guys; not fun to be too harsh on small children!
Then there is their unforgettable,
> If you also know enough data science/machine learning to design useful metrics for dashboards and workflows, that’s a major plus.
Uh, ever hear of reliability and validity? You have three hours to respond here with a relevant description! You are not permitted to use Google or Wikipedia!
(3) Ego. It appears that the A16Z specification was written by someone who has enough ego to believe that their own 'skill set' is just "the right stuff".
(4) Linux. People programming on Microsoft's platform can have done well but encountered only a small fraction of the topics in the A16Z document. The document is mostly about Linux. Sorry, guys, I'm writing in Visual Basic .NET; no, C# has too much of the deliberately 'idiosyncratic' syntax of C which I find too error prone and hate.
(5) Three Letter Acronyms (TLAs). It appears that A16Z is using obscure TLAs as code word filters. Hmm.
So, they have ORMs. Gee, what the heck is that? So, Google a little and get a Wikipedia entry and discover ORM abbreviates 'object-relational mapping'. Okay. Sure, if write the software with objects and keep the data in a relational database, then need move data between objects and tables. Given the huge variety of objects and tables, tough to believe that there is a significant 'subject' here.
Then there is their ETL. Google again and discover that that TLA abbreviates "extract, transform, and load". So that is essentially 'extracting' data from whatever original sources, 'transforming' the data as needed by the work to be done, and 'loading' the data into a data base. Okay. Sounds entirely routine. I'm surprised they didn't also list (A) LRT, look at a screen, read it, and type in what need, (B) PSS for put on shoes and socks, (C) SDP for start car, drive to the office, and park the car, and especially the crucially important (D) GSL for Google a question, select the promising results, and learn what needed. One more: WTH, waste time on Hacker News!
Okay, A16Z, two can play this game: Think of a large server farm or internet protocol network. Since this 'facility' is subject to errors of wide variety, it is important to monitor the facility for unexpected problems of wide variety, say, from software errors, hardware failures, system administration errors, security breaches, etc. To do this collect some data, say, from Microsoft's instrumentation means, HP's Mercury Interactive, what a relational database management system or Cisco box can report, etc. So, maybe 100 times a second, collect numerical data on each of 25 variables. Now we want a detector of 'anomalies', including for problems never seen before, that is, 'zero-day' problems. Such monitoring is essentially hypothesis testing. So, we want a suitable hypothesis test. Due to the fact that the multi-dimensional probability distribution of that 25 dimensional data is essentially impossible to describe at all accurately, we need an hypothesis test that is 'distribution-free'. Since the data is 25 dimensional, our test must also be 'multi-dimensional'. We want false alarm rate known in advance and adjustable in small steps over a wide range. And we want a good shot at doing nearly as well as the Neyman-Pearson result but with much less data than that result requires. Okay, A16Z, your mission, and I'd like to see you accept it, is to say how to do that. Of course, since you guys are so good and brilliant, you already know my published paper for the answer. But there is a typo in the paper. Where is it? You have three hours to post your response here.
Also needed, find a fast way to do the specified computations. I found a way, but that is not in the paper. You have 24 hours to post the answer to this part! Don't report your answer in some silly way such a running code. Instead, report your answer in clear English with any appropriate mathematics written as high quality mathematical writing, e.g., W. Rudin. Go for it, guys!
Uh, you won't find the answer on Google or GitHub! And asking around Silicon Valley won't help, either. You might want to rush off to the RAD lab project of Patterson and Fox funded by Google, Microsoft, and Sun for when they worked on such problems, but that won't help, either.
My view: Here A16Z showed that their understanding of computing is still down at the level of a junior high self taught programmer.