Stock-market investors can no longer count on the ‘index effect’
wsj.com
wsj.com
- an increase in migrations over time from the S&P MidCap Index
- an overall increase in the market’s ability to provide liquidity to index changes
What about the fact that there are more activities of additions/deletions to the S&P 500 given how much more competition there is today versus decades ago?
You will see anywhere from 15-20 additions in recent years compared to 3-5 additions 30 years ago. Their figure 1 demonstrates this well:
https://www.nber.org/system/files/working_papers/w30748/w307...
That to me just says the stock added to the S&P 500 has less time to actually accumulate said wealth to stay competitive on the top 500 list while the 1% of companies continue to reap the effect of it. Maybe that has something to do with the similar benefits of an index fund, the less activity, the more money you accumulate due to time in the market.
Once something becomes "common knowledge" in the stock market, it looses all predictive power. In this example, you could trade on the index inclusion because it observably happened, but market participants didn't respond.
Now, everyone expects the effect to happen, so it's just another thing 'baked in' to pricing.
This has nothing to do with the "time in the market" as you mention above.
Then stocks that have more "time on the S&P 500/index" are prone to survive and see higher returns also over the long term.
In practice, the market will become increasingly irrational, leading to an increasing arbitrage opportunity for active investors. There may be a period where active investment gives a better ROI, even when accounting for fees; but in the long term, the amount of passive investment should reach an equilibrium where the benefits active investment is equal to its overhead.
There is also the Wall Street Bets class of active investors. In the current market, I'm not convinced they are actually helping. But, in a world where no one else is activly investing, they are probably better than nothing.
That is outrageously wrong information.
Half the ownership of the US stock market is from households and mutual funds, with a third from the former.
The other half is mainly Etfs underlyings and pension funds.
Hedge funds represent less that 5% of the total US market ownership.
Most people collectively hold less than 1 or 2 percent of the stocks.
People and funds who have 100 billion invested won't be lazy.
One could say this happened anyway in the aftermath of the QE (a.k.a. money printing) response to the '08 crisis.
Where would those gains come from? It can't be that the index funds are matching the market and the active investors are doing better. Someone has to be doing worse in that situation.
One thought experiment is if being part of an index confers additional perceived value, the addition or removal from it index is an Arbitrage opportunity.
Imagine trading against someone who will reliably pay a premium for an index of "top 10 stocks". Price will spike for a stock that goes from #11 to 10, and drop for every stock that goes from 10 to 11.
Index investors will overpay for a newly listed stock and that money will go to an active investor that held it before the listing. Inversely, index investors will lose money whenever a stock is removed from the index.
Only buying indexes is different from only buying one index.
For example, someone willing to pay a $1 premium for an indexed stock.
It is trading at $5 today and becomes indexed, so you pay $6 for it. It leaves the index so you can only sell it for $5.
It doesn't matter how many times you average it, it is still a source of loss.
If you only have one index, going from position 11 to 10 increases the value of the stock. But if you have a lot of indices, all these effects should average out to basically zero, since that stock might be in thousands of indices, and a drop out from a single index will only have a marginal effect.
This is a long way of saying that a limited number of indices is an assumption of the hypothetical.
It would break the market if everyone invested in indexes because nobody is paying attention to fundamentals.
If there are a huge number of indexes and people are diligently picking between them, that is basically the same thing as being active investors.
- Concentration of ‘voting power’: if more investors flock to index funds, a smaller group of fund managers will have more voting power on behalf of their clients.
Theoretically, a ‘concentration of power’ could give [few] fund managers a lot more responsibility than they should otherwise have; that’s a lot of livelihoods to hold be responsible for.
- Increased crorrelation: if a [large enough] portion of the population invests in index funds, the returns of those funds will become more correlated with each other; a ripple could become a tsunami.
what are these livelihoods? They should be voting in the financial interest of the clients that own the index, which sounds fine to me.
Think of index fund investments as increasing gain on an amplifier — the more gain, the higher small signals can get boosted. An index fund is basically following the active investors and putting more weight behind each of their decisions. More passive investment money means a higher amplification factor.
If you go to a restaurant, point to the person next to you, and say "I'll have what they're having" then you'll get a meal of some sort. You may be happy with it, but who chose it? The person next to you. What if everyone does this? Who decides the initial meal that will propagate to all customers? One person.
"The researchers also explored the possibility that index changes are more predictable today than in the past, leading sophisticated market participants to trade ahead of the events. They concluded that this played only a minor role in their findings, however."
The two main factors being:
"...an increase in migrations over time from the S&P MidCap 400 index, and an overall increase in the market's ability to provide liquidity to those investors seeking to buy and sell around the time of index changes.
The additional liquidity has 'made it easier for everyone to trade, and as a result, the prices move less.' says Dr. Greenwood.
And with more stocks migrating to the S&P 500 from the S&P MidCap 400, midcap funds are selling as S&P 500 funds are buying, leading to 'a wash' in demand, he says.
"
Note that this isn't perfect - index fund A has more assets under management than index fund B, etc. But as index investing grows, the actual investor inflows and outflows are what dominate, not rebalancing moves.
This is very interesting. Do you have any examples of these funds?
I don’t put too much faith in backtesting, but several basic Buffett-style value strategies seem to beat the S&P 500 handily, with less volatility.
The last 10 to 20 years have seen a bullish round of momentum stocks negatively correlated to value loaded stocks.
Value has consistently lost money in the last decade except for short times during momentum crashes in 2018 and 2023.
Either your definition of value is inconsistent with the rest of the world, or your analysis is biased.
Considering your sentence:
> an algorithm to mass-analyze all the stocks on the US market
I guess you reinvented some variation of"low vol" or "small vs big" signal.
If you want to say you've beaten the S&P, you would have to show me a long short, beta neutral portfolio that does so using the same investment universe in a point in time manner.
I agree that predicting future price based on the current price has likely been tried to death by billions of hedge funds, but who knows, maybe they found something new that nobody has ever noticed. And, you can come up with the best stock trading algorithm in history and still not have the capital necessary to execute it, so it's possible that portfolio results don't matter.
I'm not saying anything is likely here, merely that we have nowhere near enough data to even speculate or offer advice.
You are talking about ad-hoc signals here. Of course there are a lot of them that beat the benchmark, it's the basis of active management.
But we're not talking about any signal here, we're talking about "value" - a signal so well known, studied and persistent we call it a "factor", and often decompose beta/sector/country adjusted residual returns _just_ based on a set of 4/5 of these factors.
- Was survivorship bias accounted for? If you don't have a list of stocks that includes dead issues (bankrupt, merged, or closed) then all your value stocks will be the ones that had a bad period and then miraculously recovered. Your backtest would buy those and make money, while not seeing the ones that had a bad period and then went to zero. This also means you have to track whether the stock was tradable, whether due to the exchange announcing a freeze, or the market just seizing up as participants decided not to risk much. If your strategy does shorting this is going to cause even more pain as you look for historical hard-to-borrow data.
- Was the data backfilled? This is the hardest one to drag out of data sales people, because they often just don't get it. If I have a company that reports its Q2 earnings about six weeks into Q3, is the data labelled as coming out in Q2? Because if it is, I'm backtesting on future information. Also firms that are in trouble may also have timeliness issues in getting their numbers out.
The real reason why the index effect is gone is actually simple. For years now there's been a bunch of traders who build their models around the effect. Their methods are common sense: screen the market for firms that are not yet in the index, and rank them according to how likely they are to get included. Buy a lot of what you're certain will get in, a bit less if you're less certain, and so on. The effect of this is to give you a smoothed out effect so that you can't just hop on board right before inclusion anymore.
It also seems likely that the big index investors actually hire a few of these experts to optimize their purchases. In fact I had a chat with a guy who did this yesterday. It turns out to follow an index, you don't need to buy the whole index. Stocks tend to move together so you can cover most of the principal components with just a tiny amount of stocks. You're not going to do this by buying the little stocks at the bottom of the index, you buy the big ones at the top.
On the whole, if you have the financials of all companies, and apply certain variations of the cigar-butt strategy, you get out ahead. This not only makes sense logically but also works in backtesting.
Value investing is a wide term that includes many strategies. Some of them have made quite a lot of money, very consistently.
> This not only makes sense logically but also works in backtesting.
No, it does not work at all in backtests.
That's exactly what I'm talking about: a long short, normalized, rank based on e. g EPS. A typical value signal.
Please share portfolio weights of such a signal, so that I can check the value exposure and adjusted returns.
Says who?
In my book the burden of proof is on the argument being the less in line with general consensus.
Value being a bad investment in the last decade is not an argument, it's a fact. Value is not something you find, it is a defined factor with academic consensus on the definition. You can have some leaway and build a portfolio highly correlated to value, but that only goes so far.
Claiming to have found a good value signal is then akin to claiming being able to doing consistently good factor timing on value.
This is a feat the top hedge funds on the planet would be weary to claim. Allow me to be skeptical when someone on HN claims to have done so with "basic value indicators" and found something that "prints money".
I remember him taking about discounted cash flow calculations and what maybe, supposedly, he and Munger use as the discount rate, but as soon as you apply this calculation to all 3000+ publicly traded US securities, you find that most of them are completely overvalued and have been for years. You can adjust the discount rate, but it doesn’t help in practice.
Anyone with a computer and the patience to parse financial reports can do this, so of course people have done it.
The problem is there aren’t many good investments actually there anymore. Period.
If you’ve been investing the last few years and have made returns as all of us collectively have, you know those returns definitely aren’t coming from real gains.
They’ve almost entirely been speculative returns.
I was so jaded calculating this stuff and seeing marketable securities histories across various sectors that I came to realize all the free lunch has already been eaten.
People are searching among the crumbs now.
It’s just entirely an entrenched practice. There’s too many people in that game.
I’m also a fan of the textbook guidelines from Security Analysis and The Intelligent Investor, but of course using those specific measurements also yield nothing worse investing in at the present date, and haven’t for years.[1]
Everything is way more expensive than it should be valued at, and I suspect a part of it, too, is simply that a majority of shares are on the market and have no reason to be sold lower than what inflated values are presently passing for.
It’s like owning a house knowing people need a place to live.
People need securities for retirement. There’s only also many that exist.
[1]: https://www.andrewmcwatters.org/personal-finance/#investment...
It also doesn’t help that DCF calculations are incredibly sensitive with respect to interest rates that had for some time been historically low.
It might be easier to calculate these days, but my findings from years ago informed me that these issues were beyond reading financial reports.
There’s a bigger problem of there simply just not being enough attractive companies at even fair rates.
Statistically, you can only buy earnings at so high of a price to meaningfully make a worthwhile return, and for a while now, those prices have been too high in comparison to the risk free rate.
I’m sure this has since changed due to current interest rates, but I haven’t bother to look simply because of the first problem.
It’s people repeating this idea that math has suddenly changed within the last 20 years or something that is ridiculous.
The field of finance studies how to value various assets, beginning with fixed cash flows over time, and then introducing things like risk.
Would you spend one US dollar to make a million a year? Of course, but what such investments would ever exist?
And so instruments which bare gains over time have some value, which is calculated by the means of cost of investment to returns over time.
If there are a market of such instruments, they can and will be in competition with each other.
This "things are only what people value them at" stuff is for the birds. Those other people you're talking about are actually calculating their investments. Others with such mentality will follow, not knowing what they're doing, and taking missteps when the goods are already gone.
What you think is "accepted by everyone else" is what you're seeing at face value. There are reasons behind actions. Those actions in and of themself have no meaning without context.
The mathematics of investment have never, ever changed in any meaningful way. The history of debts, interest rates, and returns have always fluctuated, but the math hasn't.
Anyone suggesting otherwise is uninformed.
There is no math that claims the price of a stock should be some form of discounted future cashflows since at least the 90s.
Since then, all models have been multifactor ones, with fundamentals (quality, value) being only one fragment of the market, sector and country residualized stock returns.
What `epups` is explaining to you is the basis of behavioral economics, which is the de facto current leading theory since its foundations were Nobel prized 20 years ago...
He doesn't do anything in private as Berkshire has so much money all their moves are public and well known. The advice is true but fits another era, read his autobiography.
> People are searching among the crumbs now.
Yeah, the cigarette butts. He had ~$10 million by age 30. Incidentally he amassed this as the the son of a congressman, had a rich auntie, and was in very good social standing in a small wealthy town (Omaha). And it was 1960.
By 1960, Buffett operated seven investment partnerships. He asked one of his partners, a doctor, to find ten other doctors willing to invest $10,000 each in his partnership. Eventually, eleven agreed, and Buffett pooled their money with a mere $100 original investment of his own.While that may be true to some extent (especially with the more popular stocks like, let's say... Tesla, the P/E ratio is through the roof, although it got a bit better in the last year or two), but I feel that stocks that pay dividends do have actual gains - in the form of dividends.
I personally prefer dividend stocks, because with the growth stocks you have to sell to get any profits or anything back, and I don't like to sell. Also, when the stuck is moving down, they usually still do pay dividends, which allows you to buy more of the stock at an even lower price.
In my opinion, it's much harder to inflate the price of the stock through the dividends, because those actually affect the company. They need to have that money to be able to pay it out to the shareholders.
Why?
Share buyback instead of dividends has almost only advantages. It's more liquid, more tax efficient.
> it's much harder to inflate the price of the stock through the dividends, because those actually affect the company.
What do you mean "inflate"?
> They need to have that money to be able to pay it out to the shareholders.
But it's the same during a buyback...
There's too much money in the game. There's more capital invested than there are things to invest in at a reasonable rate of real returns. It needs to be clawed back but it's hard an unpopular to do. What we effectively need is a temporary effective progressive wealth tax that alters the exponent of wealth distribution, and that tax just needs to go "nowhere" instead of being reinvested in government programs. Carefully though because such an action has lots of side effects. The problem is whenever you propose such a thing there will be a long list of people with a long and varied list of reasons why this or something similar is insane. It is A) complicated and B) poorly understood so the discourse is usually just useless, which is reason #1 why it is hard.
> the small rise in share value that once accompanied a stock’s addition to the index has disappeared on average over the past decade.
Quandl and Tiingo are often used for personal/retail algorithmic day traders.
This Quandl company has a terrible website. They don't say what they offer and how much it costs, all is behind either creating an account or talking to their sales department. I don't understand how they suppossedly have 400K users.
The problem is, armchair traders typically commit a few classical blunders:
1. Accidentally used future information to make trading decisions in backtesting. Obviously a crystal ball would make for GREAT alpha. And while you might think it's easy to avoid doing this, there can be some very subtle oopsies that are not apparent at all to even a somewhat sophisticated analyst that will catch you out. This was a common oopsie I would see our analysts do, sometimes only even becoming evident after the research team had vetted and idea and they hand it off to me to implement and I'd be sitting there in the data flow and have to tell them "yeah this data item you used, it's latency sensitive, did you make sure you were using the appropriate network path when you keyed into the dict or did you just hardcode the common one - because if you hardcoded then you're using a crystal ball for all the other network paths".
2. Assuming that if you see a price, you can trade at that price. Even HFTs can't just go get any ole' price they see.
2.5. A tangent to 2 is that your own trading activity can impact prices. Not important if your alpha is long term but if it's short term then this can be significant.
3. Ignored trading fees, and I don't mean brokerage fees - I mean SEC fees, etc that are usually very small for most retail trades so most brokerages don't even show you them (especially because depending on exactly how they executed your trade they might be different than you'd get if you just sent them to an exchange).
4. Not taking into account tail risks. Look at how many retail investors got blown the fuck up when XIV died [0]
[0] https://www.forbes.com/sites/jimcollins/2018/02/06/it-works-...
Trading is hard. There's literally armies of ridiculously sophisticated actors as your market participant peers. I always used to joke that the federal budget is to defense spending as our nation's brainpower is to finance. (We probably throw WAY too much %-wise at it, but damn do we get a world class product as a result.)
I prefer to repeat to myself the mantra of: "Time in the market beats timing the market."
People thought they could beat the market by timing buys of stocks that were likely to become part of indexes.
If there was any alpha in this practice, it’s been sucked out of the market at this point. The anticipation of this has already been priced in, so there’s no remaining advantage in trying this strategy.
I'm also guessing that many have algorithms that check for stocks that fulfill the criteria to be added to the index.
I think but am not sure that the economy can expand by shifting more “income” from capital to wages which would be bad for stocks. Think 1950s.
If the expanding economy is caused by a lot of disruption from new businesses it could also be bad to hold indices weighted towards incumbents, this seems the most likely to happen of the three cases. The disrupters don’t necessarily need to be big, just enough of a threat that investors think future growth or earnings of incumbents are threatened.
To see all WSJ submissions to Hacker News:> https://news.ycombinator.com/from?site=wsj.com