Hacker News Reader

Comment by somethoughts | Hacker News Reader

I think its helpful to realize that the S&P 500 is not some static index of companies whose selection has been on auto-pilot but actually a fairly actively managed index whose selection algorithm is continuously being updated and refined. There is a not insignificant human judgement component of how to define the selection/weighting criteria.

In some senses, behind the scenes its likely similar to the Twitter/FB/Netflix algorithm - at some point there is some editorial opinion being inserted via criteria and weighing of those criteria.

"Some years ago Bill Miller, then a well-known Legg Mason portfolio manager with a stellar record of beating the market, noted that the S&P 500’s track record of being very hard to beat suggested that active management can succeed and that the Index Committee were actually good active managers."

https://www.indexologyblog.com/2014/08/07/inside-the-sp-500-...

View on HN

Yes there's an index committee comprised of humans. But the index methodology is public and when there are tie break decisions needed, they're fairly predictable.

Another thing to think about is that most of the outperformance of the index is driven by the largest companies. But not because they were added when they were their current size. But because of the buy and hold approach of getting them in when they're still on the small end of the large caps. There was a good recent paper on this concept, showcasing that most of the big index returns are attributable to only a few stocks. Similar to how most of global market returns are mostly just US.

I think you mean that the committee comprises humans.

Definitely agree the algorithm is likely pretty well documented but as noted in the blog it is adapted in realtime.

Here's the current breakdown:

Apple 7.08%, Microsoft 6.20%, Amazon.com 2.62%, NVIDIA 1.87%, Alphabet 1.84% Berkshire 1.65% Alphabet 1.61% Tesla 1.44%

which if it was an active manager would be highly opinionated.

Really? That's essentially just the market cap weights, which is the most default thing you could do.

There's a slight deviation because it's float-adjusted and I'm assuming their marks are at some frequency, but those numbers aren't mysteries.

I think the point is that the divergence in market cap is so large that it’s become equivalent to a highly opinionated active manager. Any fund that’s allocating over 13% to just two companies seems a bit risky, but that’s what you’re getting with Apple and Microsoft here.

I don’t think it’s as opinionated nor active as you’re making it out to be.

Investing proportionally more in a huge company than you do in the 500th largest eligible company seems much more passive than active to me.

If two S&P companies were to merge and everything else remained the same, I’d think that having the combined amount invested in the combined company to be more sensible than cutting your investment proportion in half merely by a merger.

Or if a company you’d invested in doubled in value, I’d rather hold the constant number of shares rather than sell half of them to bring my exposure to the winner down. I’d also view that as passive and un-opinionated. Selling half feels more active and opinionated than simply holding.

I’m sorry, but this is just not true. Active managers are measured by their deviation from the S&P (i.e. a beta 1 manager is extremely passive). This index has also always been concentrated to some extent, so not sure it’s any kind of recent change.

I think maybe you are talking about risk, but it’s a completely separate idea.

unrelated basic question: Berkshire itself is a conglomerate that owns oprtions of S&P companies like Apple etc. So is this not double dipping?

I was confused by NVIDIA having a similar weighting to Alphabet despite having less than half the market cap. But notice that Alphabet appears twice. (GOOG and GOOGL.)

Is the algorithm "select * from companies order by market_cap desc limit 500"? At the risk of oversimplifying, there should be quite little to debate, and when some hair-splitting occurs, I trust a committee of reasonable humans will make a reasonable choice.

The algorithm is a little more complicated than that, as various judgements have been added in, but always with specific rules. When there was a trend of tech companies going public with dual classes of shares leaving founders in control forever (e.g. Zuck with Facebook) they announced that any company which went public after date X with dual classes to prevent the economic owners from exercising control would not be allowed onto the index. Similarly, there are rules about profitability that, IIRC, were added after the first dotcom bubble to try and keep extreme bubbles from mucking up the index (I think it's that you need to have two consecutive quarters of profitability, but I don't remember the details).

Also, the market cap of company 450 and company 550 are close enough that recent market performance can flop ]back and forth fairly regularly, which the S&P has further rules to try and limit, though again I don't remember the details. Again, they try to be fairly rules based, but do exercise some discretion in creating the rules.

You have to adjust for free float market cap. Companies need to be US incorporated. You have to be aware of share classes. For example both GOOG share classes are in the S&P.

When it comes to tiebreak decisions, there may be 5 or so potential adds when there is 1 delete. There may be a focus on evening out sector exposures, preferencing underweight sector adds. (Just my guess)

That's interesting. I was contacted by a personal money manager looking for clients and he explained his philosophy as sampling 30-50 stocks from the top end of the S&P 500. He had outperformed the market during a bull run, which I attributed to the increased risk of a smaller portfolio. I didn't invest because I figured his fund would underperform the index during the next bear run for that same reason, canceling out the gains and moreso with fees. Maybe there actually was something to his strategy; I haven't followed up on his performance.

Owning a handful of shares does capture a significant amount of the beta for a market.

However with very significant profitability outliers, such as power law distributions, then you have to own all the shares to ensure you don’t miss out on the one or two shares that make up most of the gains. Venture capitalists talk about this a lot.

Historically the same issue occurred with market timing: most market gains are made during only a few relatively short unpredictable periods. You need to own shares for a long time to ensure you are in the market when those few unpredictable ups occur.

IIRC many active managers beat the market/S&P 500/benchmark before fees. It's almost impossible to beat after fees because the fees are too much of a drag.

That’s not true.

The SP500 index might be a little more complicated than just the 500 biggest public companies. When Tesla was rejected despite meeting criteria, it was odd. However “edition” impacts less than 1% the actual value.

If you can compare VOO (Vanguard sp500 tracking) vs VV (Vanguard 566 large cap), it’s the same: https://www.etf.com/etfanalytics/etf-comparison-tool/VOO-vs-... (20 basis-point difference)

> * There is a not insignificant human judgement component of how to define the selection/weighting criteria.*

Not wrong, but the rules are relatively fixed and known ahead of time and somewhat more deterministic compared to the decision-making process of most active funds.

It should also be noted that the S&P 500 isn't the only index. The Russell 3000 and Wilshire 5000 try to cover "all" publicly trade companies in the US:

* https://en.wikipedia.org/wiki/Russell_3000_Index

* https://en.wikipedia.org/wiki/Wilshire_5000

But "passive" investing does exist on a spectrum:

> The terms passive investing and index investing are often intertwined, but they are not exactly the same thing. Today’s guest is Adriana Robertson, the Honourable Justice Frank Iacobucci Chair in Capital Markets Regulation […].Adriana is interested in index investing and, in this episode, we hear her views on whether or not index investing is passive. Hear facts from her paper on the S&P 500 Index fund specifically, and all of the reasons that it's not passive, as well as some of the issues that are potentially arising from the creation of so many indexes or so-called passive investments. A more recent paper by Adriana, published in The Journal of Finance, surveyed a representative sample of U.S. individual investors about how well leading academic theories describe their financial beliefs and decisions, and Adriana shares the differences in something like value growth from an academic perspective versus a real-world perspective. Find out how investors can go about evaluating the performance of their portfolios and what they should be looking for when deciding which index fund to invest in, as well as why index funds aren’t a meaningful category anyway, factors from Adriana’s surveys that might influence investor’s equity allocation, and the trend towards indexing and whether it will overtake active portfolios. Tune in today for all this and more!

* https://rationalreminder.ca/podcast/133

Agreed. I have ruined so many dinner conversations over this. You can't beat the SP500, it is the only free lunch. It is truly depressing when you think about it - you just feed the machine. The American millionare class is a group of 401k holders. [Oh. and they made 54.96% in the last 5 years]

> The American millionare class is a group of 401k holders.

What do you mean by this? That pension saving make up the bulk of wealth in the US? Because from what I can gather that's not correct.

Your best odds of being an american millionare is pushing all you can into a 401k which purchases sp500. Most american millionares are not business ninjas. They are teachers and firemen pushing into their 401k, they also don't make much money. "Only 31% averaged $100,000 a year over the course of their career."

What happens is the ridiculous return of the sp500.

Sorry, about this link in advance: https://www.ramseysolutions.com/retirement/the-national-stud...

Being in the millionaire class in the US isn’t that rare[1] and very often is achieved by real estate and retirement savings.

[1]-It’s about 9% of American adults.

> It’s about 9% of American adults.

How is this counted wrt spouses?

Yes but the vast bulk of wealth is owned by a very small minority. That's who people are generally talking about when referring to the "millionaire class".

I think he's saying that 401ks have made a bunch of regular people into millionaires, because of the performance of the S&P 500.

Yes, easily. Buy an emerging market etf. Will outperform SP500 over the next decade.

Over the next decade, but not the next year? Are you so confident in this that you've leveraged and bought options to maximize returns?

I'm all in on emerging markets personally.

Interesting metaphor. Usually, one doesn't buy into a poker game with one's entire bankroll, so pushing all in is relatively insignificant.

It's not really fair to ask someone to leverage themselves to display their conviction.

It's not really fair to make an assertion that one asset will outperform another without giving a measure of confidence.

I'm not asking that our advisor take leverage, but merely asking whether she/he has, to gauge confidence in the investment thesis.

Forward looking statements made with such confidence tend to be wrong. I take it you will personally make whole anybody who follows your investment advice in case you are wrong?

But you don't have an issue with the statement "you can't beat the sp500". Just calling out American self centeredness here.

Yeah but the Russell 3000 tends to slightly beat the S&P 500, and it comprises 98% of the market.

That's just the size premium.

They are not actually selling stock that's not performing well and buying stock that is, though, are they? It's very hard to beat their performance if they can pick winners at any time and just discard losers without any actual loss.

.. and it's "THE" S&P 500 ... popularity surely has an impact on whether an investor will simply choose to invest in a member of the '500.

> Index Committee were actually good active managers.

It's as if researchers complained that placebo is too therapeutic.

archive unpaywall link: https://archive.is/20230308071523/https://www.ft.com/content...

Reply on news.ycombinator.com