Hacker News Reader

Comment by habosa | Hacker News Reader

In the chart, this is how the fund has compared to an S&P index over the past 5 years:

  * 2013 - 5% over
  * 2014 - 15% under (and negative overall)
  * 2015 - 30% over
  * 2016 - 10% over
  * 2017 - About even

One exceptionally strong year, and pretty uneven otherwise. Hardly proof that these biology-derived algorithms are the secret to market-beating returns.

View on HN

We have no idea what the beta of the fund is; it's difficult to talk about performance without knowledge of the fund's underlying risk

The other problem is scale - this is pretty evidently an advertising attempt in order to raise more cash. It's a lot easier to return 20% on, say, $100M AUM than it is on $1B AUM.

To be fair -- these caveats are true for almost every fund you hear about. I do think that's kind of my big issue with the article though, that this fund is just like every other fund; it's intrinsically an actively managed portfolio with the same shortcomings as any other. It's just got a good publicist!

I don't understand why 100 mil aum is different from 1 bill aum. couldn't you just divide the billion amongst 10 groups that are Chinese walled from each other? then there'd be no problems of market moving or depth or anything.

Your strategy means that you need 10x more working algorithms. Harder to find 10 than 1.

I agree, I don't believe this would pass a test for statistical significance (of outperforming the market indices).

Unless there is a way to break this data down much more finely (by 6-day window, by individual asset) in which case it's possible there's a high p-value but low-power effect. Even so, if there are more than 100 experimental firms, a single one with p-value .01 isn't evidence enough to jumping to big conclusions.

Also the selection effect of us just being aware of it.

It seems like you could probably insure some great press coverage by starting several funds with wild-ass prediction mechanisms that don't change much. As long as you pick enough investments to roughly mirror the market, you can spin all the winners as incredible breakthrough techniques.

Heh, it's like that scam where you email people your predictions for who will win each football game for a particular team, where different groups get different winners. Each week some of the emails are right and some are wrong. After six weeks, 1.5% of your original folks will have seen six correct predictions, at which point you ask them for $1000 to see the 7th prediction which they are likely to pay for since "you're so accurate!".

Hah, that's excellent! I'd never heard of that specific trick, but survivor bias does seem like one of the best ways to fool scam-aware people into trusting you.

I had a b-school class do a similar exercise, in the context of the debate between passively and actively managed funds. Everyone in the class was asked to stand up, pull out a coin and flip it. If you flipped tails you sat down. Then the remaining students flipped again, repeating until there was only one person left standing. At which point the professor "interviewed" the student, asking what her method was and how she became such a skilled heads-flipper. Lots of parallels in the real world where luck is a dominating factor (stock picking, choosing a "rocket ship" company to join or invest in, general business success), yet our tendency is to seek out a cause or reason.

I like the exercise. But from a practical pedagogical standpoint (in the unlikely scenario I ever teach a class probability or survivor bias), what would be a safe point to stop the flipping? 3 students left standing? 2? What if all of the students still standing all flip tails simultaneously?

Oh, that's a good question. I'd stop at 4, or certainly anywhere under that. 4 flipping is a 1/16 chance of lesson failure, which isn't bad but is a bit uncomfortable.

Maybe you'd weight it based on class size? 4 left isn't bad in a class of 100, but it's a bit high in a class of 10.

I like this. Unfortunately today, you'd be unlikely to find students with coins!

You flip a coin, have students bet on which side the coin will land.

If the winning techniques continue to generate greater risk-adjusted returns than the market for a significant amount of time, you haven't really bypassed the whole, "finding alpha" bit. Unless of course you've spun up a truly absurd number of funds, but I'd find that harder to believe than just beating the market in the first place given the capital requirements.

"Fuzzing" for alpha that way would be really inefficient.

I think I was a little unclear. I'm not actually proposing fuzzing alpha, just capitalizing on survivor bias. Basically my idea was that if you're going to spin up a 'normal' managed fund, you'll average tracking the market, with some chance of success or failure beyond that.

If that's all you're doing, you might as well go all-out on claiming to have secret sauce that produces your alpha. I'm not expecting you'll get any, just that you might as well have a media-friendly way of picking if you don't have an actually useful one.

Also there's a question of how many new hedge funds are using machine learning. If you have enough different attempts, one of them will beat the market by sheer game of chance. Picking out their specific quirks afterwards (algae etc.) is just survivor bias.

The whole industry is based on investors not understanding survivor bias and quite often traders not understanding hindsight bias.

> If you have enough different attempts, one of them will beat the market by sheer game of chance.

How do you define an "attempt", and how many "attempts" do you need to beat the market (consistently) by chance?

Get 25 monkeys to throw darts at S&P 500 stock symbols on January 1st of each year. The chance that at least one monkey outperforms 5 years in a row is about 55%. (The target areas need to be proportional to market capitalization etc.)

That's if you have one trade per year. How would you model this if you have 25 monkeys throwing, say, 300 darts at the board per day, for every day that the market is open (252 days), for five years?

If you're going to quantify survivorship bias, you can't use entire years as data points, because that doesn't properly represent the amount of activity that occurs. We should reason about each event, because if consistency emerges on an event basis we might not even need more than one year for our sample. The decision-making that is being empirically examined here (i.e. acumen capable of beating the market beyond chance) ostensibly functions on trading events, which means years are not the correct data point to use (and will provide an incorrectly pessimistic sample).

> That's if you have one trade per year. How would you model this if you have 25 monkeys throwing, say, 300 darts at the board per day, for every day that the market is open (252 days), for five years?

If the data is reported on a yearly basis then it's pretty much the same thing.

No it isn't, because each firm doesn't have a 50% chance of beating the market each year. Unless you're postulating that that is the case, it's not at all the same.

I can quibble about the odds of each individual trade resulting in profit or less being binary, but for the sake of argument it'll do. But a 50% chance of beating the market each year isn't supported by anything.

The grouping of data reporting doesn't suggest anything about the underlying data if it doesn't also share the same probability distribution. The trades are the events which determine if a fund will outperform on an annual basis, and we can group those trades by day, week, month, year, etc.

Strategy A: buy one random stock per year.

Strategy B: buy two random stocks.

If the expected payoffs were mathematically different then you'd have an arbitrage opportunity. Then apply induction.

Well 1.05x0.85x1.30x1.10x1.00=1.27

Wow, so they are ~27% off the mean!

The part that isn't clear is how this compared to typical performance variation - then we can run our p-tests. But I suspect its pretty good, given the news worthiness of the article.

Wouldn't 1.27^0.2=~1.05 be a better indicator?

Yes. 1.05 over the mean.

Also for the actual return:

    2013   +22%
    2014    -2%
    2015   +33%
    2016   +22%
    2017   +10% (til June)

(1.22 * 0.98 * 1.33 * 1.22 * 1.1) ^ 0.2 = 1.16

Or for the 4 whole years:

(1.22 * 0.98 * 1.33 * 1.22) ^ 0.25 = 1.18

Also, hasn't been tested under bear market conditions... I'm skeptical that past returns prove future gains here. I can take any small subset of something and say its beating the market.

Reply on news.ycombinator.com