There is Little to No Relationship Between Financial Runway and Startup Success | Hacker News Reader

There is Little to No Relationship Between Financial Runway and Startup Success

cbinsights.com

53 pointsasanwal37 comments

The author is claiming a one independent variable regression explains something. Starting with ommitted-variable bias [1], I don't think this shows much. The author only looks at

   high-tech companies which had raised a Seed or Series A round AND which had 
   exited and where an exit valuation was available

a bad analysis does not create meaningful information. At bare minimum, to gain any real insights, you need to include startups that failed.

[1] http://en.wikipedia.org/wiki/Omitted-variable_bias

The analysis is only looking at whether there's a strong negative correlation between initial funding and startup success, using exit valuation as a measure, as this was Fred Wilson's assertion.

Of course, this is not meant to be predictive but rather meant to dispel the notion that a strong negative correlation exists between these two variables.

I don't see where Fred says that exit valuation is a metric. His blog post on the subject never includes the phrase "exit valuation".

http://www.avc.com/a_vc/2013/09/maximizing-runway-can-minimi...

He is a VC, and a good one, so success = exit valuation by definition. This is implicit when a VC is talking about success.

Fred talks about he defines success here - http://www.avc.com/a_vc/2010/06/how-we-measure-success.html

"We are financial investors and we do want to see our portfolio companies become valuable."

No, it really doesn't imply that...

Sorry, but I don't understand this comment. If exit size is not one of the primary metrics VCs look at and how they define success, I'm at a loss for what would be.

More specifically, how do you think USV and Fred define success? Fred says that financial returns (hence exits) are important on his own btw [1]

[1] http://www.avc.com/a_vc/2010/06/how-we-measure-success.html

Exit size doesn't matter on its own - you have to take into account how much equity you own. 40% of $100 million exit is better than 10% of $300 million - as an example. I would describe the first scenario as more successful than the second.

If you paid $50 million for the 40% and $5 million for the 10%, I think you would be hard-pressed to describe the 40% investment as more successful.

I am taking this from the founder's point of view, where no money is paid to buy the equity. You highlight the good point that it depends on the context and point of view - who's success is being talked about.

To further my other comment - I'm a regular commenter at AVC.com and Fred generally talks from the perspective of what's best for the founder, for the entrepreneur. This is why I am making these insights.

First, he doesn't say "strong" correlation.

Second, your analysis doesn't work without including zeroes. Pretend all the failures raised a lot of money. Your line would point down.

Your analysis was marginal and you're not taking the feedback very well.

Thx for the comment. We are updating the brief with all the zeros as we have the data.

You are right that the line would point down if all failures raised a lot of money at the seed/Series A stage, but that's not the case. This intuitively makes sense as only a small select group of companies/founders can raise large initial rounds (serial entrepreneur, amazing traction, etc) and most will raise smaller sums.

But thx for comment. Update with zeros coming soon.

You're making strong assumptions about his methodology vs a guy that is only looking at public ally available exits. I would be remiss to say that his data most definitely supports his position.

To clarify, Fred's post has no data - just a statement of an inverse correlation between funding and success.

and since it does nothing like dispel the notion (see, again, omitted variables)... what exactly does this contribute again?

Not to mention that exit valuation is a dubious metric of startup success to begin with. It also excludes startups that don't exit. Airbnb, Dropbox, etc wouldn't be included.

I think you need to look at this from Fred Wilson's perspective as a VC. Exit is the only the measure of success. It is how he and every VC is ultimately incentivized by their LPs. The more exits and the larger those exits, the more money the VC makes and the more likely they'll have LPs invest in their next fund.

Of course, one can argue that startups should be not be in it for the exit, but when you take VC money, that is what you are signing up for.

Fred talks about he defines success here - http://www.avc.com/a_vc/2010/06/how-we-measure-success.html

"We are financial investors and we do want to see our portfolio companies become valuable."

As a VC the startups that don't exit are crucial to the analysis. For their purposes they should be included with an exit value of zero.

Say the startups with the highest (or lowest) funding all die before exiting -- that's a really important thing to know. That'd affect the regression outcomes (not to mention the decision-making process of everyone involved.)

Great point. We have this data and will update with that plot. Thx for the comment.

Who cares about startups that don't exit? We aren't here to build companies, we are here to exit and make lots of money! :)

I have a feeling that if you're a startup that took VC money, you very much care about an exit. Whether you like it or not.

Interesting. As a nitpick, this:

  Specifically, we looked at high-tech companies which had raised a Seed or
  Series A round AND which had exited and where an exit valuation was available.

skews things, because startups that exit for less than the amount of funding received are less likely to have their acquisition price reported publicly. It eliminates points from the bottom right of the graph, which could be responsible for the apparent positive correlation. Perhaps Fred would be right if those data points weren't excluded (and he does have access to those data points for his own companies).

Good catch ... this is eliminating an adequate amount of the relevant data.

You have to understand first the context and that is the biased view that Fred has based on the companies he / Union Square Ventures invests in - and for where he'll see the most data from.

Also, you have to define success - and success for who? If you take a larger Series A, that likely means founders are diluting more - and possibly previous investors. The idea being that if you can take as little as possible and find other means to grow and continue to develop product - other than throwing more money at it - then you'll be more 'successful' - whatever success means to you.

If how much you sell a company for is your what you care about, and not how much equity you have out of it - then cool - but 40% of something worth $100 million, is better than 10% of something worth $300 million.

It is good to look at the data, but Fred Wilson's assertion is not necessarily wrong. Data notwithstanding.

There are a large number of reasons why getting too much money is bad for a company. See the "Don't raise too much" section of http://www.paulgraham.com/fr.html for some of it. If you want a much more thorough analysis (albeit in a different context), the negative dynamics of too much money are studied in detail in The Innovator's Solution.

That said, investors like Fred Wilson are aware of this risk. Therefore they will attempt to avoid investing too much in companies that can't handle it. Thus the fact that a company received more money means that, in the judgement of investors, it was a company that could absorb more money. If the investors do their job well, you would therefore expect to see little to no correlation between the amount invested in a startup and the subsequent success of said startup.

The right analysis is impossible to do. But it is to compare what a competent investor (eg Fred Wilson) thought a company could handle, compared to what it got, and see if there are correlations there.

Let's go to the original post [1]. Fred says "To my mind, maximizing runway is not the game startups should be playing. Getting somewhere fast is the game they should be playing." This is consistent with the VC playbook. They invest in high growth companies and want to fund expansion, not an extension of "As is".

Let's look at a few issues with the OP's analysis:

1) As others mentioned, there is a survivor bias.

2) Runway should be measured in months, not in millions. Size of funding to log size of exit is the wrong metric. Months of runway to IRR of exit is the better comparison.

3) I forgot what #3 was.

Even when the counter-argument isn't great, I still like the discussion.

[1] http://www.avc.com/a_vc/2013/09/maximizing-runway-can-minimi...

Thanks for the comment.

1) We include asset sales/talent acquisitions but yes, private company data is imperfect. That said, we have the best in the biz (highly biased)

2) Runway in months and millions is semantics. If you have more millions in the bank, you have a longer runway in months almost by definition. IRR of exit - not sure I follow how that is better (and more importantly, an impossible metric to get at scale for private companies)

3) Agree :)

Thanks for the reply. On #2 - isn't runway money/burnrate? I always viewed it as measured in months. "We have 12 months of runway" versus "we have 24 months". I mention this because the original Wilson post was encouraging people not to stay too lean purely to increase the runway, implying the runway could be variable for a given amount of money.

IRR data is semi public, no? Isn't it possible to see how much a company gave up in the A round by comparing valuation to money raised? Then back out the IRRvat the IPO?

Very interesting graph. But this is kinda obvious - success of startup is really about team, product and market. The way how company raises money (or not) it should not really matter. If a way how company raises money really matters, then VCs will care less about team, product, and market and more about who is investing with them and how much money is needed. Which is obviously not true...

Statistically speaking, the interpretation of the regression is "there is no correlation between the duration of financial runway and a startup's exit valuation."

It could be whether or not you have a runway could impact whether or not you have an exit.

In addition to the discussion here, you can also follow the discussion on fred's article here. https://news.ycombinator.com/item?id=6408318

It'd be helpful to see this study normalized for ROI over time, which is arguably a better measure of VC success than exit size.

Doing so might very well make Fred's contention hold up.

I'd bet on Fred Wilson being right. As a VC he has access to raw deal data that this author does not.

Does he though? Sure, for USV investments it would make sense. But I'd be surprised if other VCs shared data on their failed investments with him or his firm though.

I can't imagine a more skewed data set for fundraising and exits than news and PR. I'd much rather look at the raw data for a single firm in the absence of industry-wide data.

It seems like someone misinterpreted here. This plot specifically excludes startups that went bankrupt or were otherwise terminated, which is what I would consider "failure".

As best I can tell, this plot is "runway vs. degree of success" rather than "runway vs. rate of total failure", which I think would be the more interesting plot.

Survival bias much?

Reply on news.ycombinator.com