Hacker News Reader

Comment by sfblah | Hacker News Reader

Unpopular opinion, but I suspect this is actually how a lot of hedge funds have historically outperformed the markets. The amount of nonpublic data needed to get an edge is surprisingly small.

View on HN

My mostly uninformed hunch is that much of the fancy "public information" analysis that hedge funds claim to do to have their edge is actually parallel construction for insider tips. I've heard it claimed that some funds have flown helicopters over oil storage fields to measure the shadows in the tanks to know how much oil is stored, or bought satellite photos to analyze the parking lot use of major retailers during the Christmas season.

These things sound way too specific to just perform on a hunch, and even then, you might get the analysis wrong if you don't have the insider tip to check your math.

But it's probably enough to keep the SEC investigator from having a case unless they find the insider link directly.

> I've heard it claimed that some funds have flown helicopters over oil storage fields to measure the shadows in the tanks to know how much oil is stored, or bought satellite photos to analyze the parking lot use of major retailers during the Christmas season.

This is really common and basically table stakes for sophisticated firms now (those that invest in relevant sectors). They track everything from parking lots to freighters and oil tankers to crop yields via satellite imagery and flyovers.

It's gotten cheaper with drones so more important areas like major ports might get daily fly bys to track containers and boat traffic. Some have static cameras pointed at these things

The use of aerial imagery for institutional asset tracking is older and much more widespread than we'd think.

Back in 2005/2006 my university maths professor (in Finland) had an established side gig in the US. He had polished a process to manufacture aerial drones - still called UAVs at the time - and had put together a fairly slick software pipeline to combine GPS tracking with digital imagery. One of his longer-term clients at the time was Harvard University; they had contracted his firm to get routine data on how their land endowments were doing.

An associate professor at the CS department has had a similar thing going on since ~2005. His company does drone imagery for land owners in Finland. Rather surprisingly a notable fraction of his business at the time was coming in from families and corporate offices wanting "just" nice looking shots of their various farm buildings.

At the time the university in question was rather well known for their computer vision unit.

Try 1910's. Reminiscences of a Stock Operator describes Jesse Livermore collecting unique public data on shops and use for trading. The founder at my first trading job gave me this book to read, and I am eternally grateful.

Don’t forget the easiest dataset to use: they buy consumer transaction data directly from Visa and Mastercard.

With that in hand it’d be pretty easy to know if, say, Apple will beat or miss revenue for a given quarter.

> With that in hand it’d be pretty easy to know if, say, Apple will beat or miss revenue for a given quarter.

Wow, seems so easy! That's not how the market works though. You've got to know if Apple's going to beat the analysts' expectations of how Apple is going to do. All the analysts have access to that data and that's table stakes.

Apple's guidance on revenue is only like 30% of the equation.

Typically, because they make so many bets, a hedge fund needs to be only 53-54% accurate with their predictions to make money.

The stock of every company will move a decently large amount on the day quarterly revenues are published, whether it's Apple or a smaller firm. You can make a lot of money via options maturing on such days. When you need to be right only 53% of the time, these signals _really_ add up.

>Typically, because they make so many bets, a hedge fund needs to be only 53-54% accurate with their predictions to make money.

Wouldn't that imply that every bet the fund makes is roughly of the same $$ value? I would figure more senior traders or more successful ones would vary the size of their bets relative to certainty or uncertainty of an event outcome (could be quarterly earnings, a political/geopolitical event, etc)?

Don't think of it like buying a stock on the spot market like retail investors typically do. Many hedge funds place their bets by using the options market, which limits the downside on a bad bet (see e.g. the butterfly strategy [1]).

The size of the bet also depends on a lot of factors. Many hedge funds don't work like monolithic entities, rather they have a bunch of portfolio managers (PMs) who have their own allocated pools of assets from investors. These PMs decide on placing their bets based on lots of different things like their liquidity position (e.g. you don't want to lock up your cash in one position even if you're sure it'll make money when there's a risk of being margin called because of another trade) in addition to signals such as the one dicussed above.

[1] https://en.m.wikipedia.org/wiki/Butterfly_(options)

The only thing I can speak to is crop yields. Satellite imagery S flyovers won't really teach you a lot outside of a binary yes/no r field is alive.

The technology farmers use teams yields to the square foot. I'm betting you can buy that from John Deere and others. That's probably where they get that?

I don't know how effective the project was because I left before it finished but we were working on using the normalized difference vegetation index, normalized difference water index, and a bunch of other data points to try to predict crop yields.

Farmers might get that kind of stuff from flyovers for the extra resolution but that was cost prohibitive for our purposes except when validating data from the satellites.

Yeah good luck getting accurate NDVI measurements down to the square foot from satellites.

This year in university I took a course where I had to imagine creating a startup and we ended up mocking up an agritech business that dealt with drones and satellites data. We discovered that Sentinel 2, a European satellite, has a resolution for ndvi that goes as low as 10/20m. This improves nasa's modis sitting at a resolution of ~250m.

Furthermore, this is free. There are companies providing paid images with higher resolution from their own satellites (Constellr comes to mind). I'm sure the resolution isn't going to be exact to the square foot, but for some cases you probably don't need such a high resolution

Luckily corn fields go on for miles so we didn’t need that resolution.

Fields vary wildly within relatively small spaces. Just because it looks uniform doesn't mean it is uniform.

Our 100 acre bottom field this year averaged 274 bushels of corn per acre. Some places were around 360, some were around 50.

If you're trying to predict crop yields to beat the market, granularity is wildly important.

> major ports might get daily fly bys to track containers and boat traffic

Shipping traffic has transponders these days, no different to air. Don't even need drones.

That actually is public information. I know it's hard for your typical member of the public to obtain, but the key thing is that it didn't wasn't communicated directly by someone working at the company. That's legitimate and no parallel construction is needed (except to mislead your competitors – maybe that's the real motivation).

> These things sound way too specific to just perform on a hunch, and even then, you might get the analysis wrong if you don't have the insider tip to check your math.

This is suggested as the parallel construction for the actual insider info.

Ordering satellite imagery and counting cars is just a weekend project. The last time I looked at ordering imagery, the main obstacle was the minimum order size, so it'd actually scale better for monitoring every store car park than for looking at a single car park.

So if Macy's parking lots have 11% more cars than the same time last year, is that a buy or a sell? Are people actually buying more, or are they more cash strapped and spending more time looking for value?

You’d have to have historical data to see if more cars mean more spending.

It’s an indicator they’re getting more traffic. Which you can then feed into your model to decide if it’s a buy or a sell, based on all other data.

For instance, is the stock and/or expected earnings > 11%, while traffic seems to be only 11% - or vice versa.

What if Macy‘s parking lots have fewer cars but they’re selling more and more online now?

How busy are the car parks by their dispatch center? are the cars staying longer because people are working overtime? how many UPS trucks are visiting?

Ok fair, so you need to have a model for every one of their revenue streams.

you buy data flow data from ISPs at all tiers, so even though they're encrypted, knowing how much traffic is going to Macy's.com vs JCPenney.com gives you information you can act on.

We know this is being done, because of reports that say Netflix is X% of Internet traffic. The undredacted reports from those same data sources have much more detail. It's also why some apps that don't appear to have any business model are actually quite valuable.

2024 answer: just train a predictive AI with that rarely measured data and avoid thinking about the innards of the black box.

In terms of parallel construction, can you tell the difference between insider trading and confident-sounding tips from WallStreetBetsLM?

I know a guy who's been making and winning steep-odds bets on baseball games based on wind patterns at stadiums on game days.

The SEC could still have a case - timeline, corroborating intrigue, etc. - but as an underfunded revolving door, there's little pushing them to even try to hold these firms accountable. The exchange between a congressperson/senator and an SEC rep a few years ago - "How many cases do you take to trial?" "It's more efficient for us to settle," or something along those lines - was pretty damning. (I apologize, the exact details escape me and every single search function on the internet has apparently been degraded into uselessness.)

The helicopter and satellite tricks though are legal, right? They do not involve any leaks from insiders.

Personally I think those things sound like the natural result of ex-intelligence agents wind up working in finance, sort of like the advertising boom seen from the ex-WW2 propagandists.

>> parallel construction for insider tips

That is almost certainly the case.

I recall learning this lesson pretty early on. In high school, the econ teacher ran a stock trading game - you get some starting capital, you make trades, at the end of the quarter whoever had the most got some kind of reward.

At the time we had a family friend that was working for a company about to announce a stock split along with a very good earnings report. He told me when to go all in on that stock, and i did exactly that. The day after that split my portfolio had more than doubled, beating the class record by a significant margin. Said record stood until the teacher retired.

Not sure if it was the lesson he meant to impart though since I think most took away from it that to win you need to lie, cheat and steal.

I knew someone who won a similar stock trading contest held by a radio station about 30 years ago. He put all his pretend money into a low volume penny stock. Then in real life he bought enough of that same penny stock to raise the stock price substantially. He'd somehow calculated that the cash prize would exceed the cost of manipulating the stock. IIRC the stock price got another boost when they announced his winning trade on the radio, enabling him to make a tidy profit.

That's exactly how clients beat bucket shops as well. If they don't set their risk limits low enough and fees high enough, you can profit by manipulating the underlying markets.

I won a similar contest by being the only student who never bothered to log into my trading account, thus keeping all my assets in cash by default. The market had a down week, so everybody except me was in the red.

I think the lesson, which has served me well, is to not make short-term trades.

When I studied abroad, I lost my debit card and had to get it mailed to me internationally. I was down to nothing and spent a week living off of rice and furikake.

What saved me is one of these trading contests for my school's business club. I signed up for it and dumped all of my "money" into playboy and forgot about it. Turns out they won some big lawsuit and the stock spiked just in time. First place was a $200 dollar fine. I had to have the club president spot me the train ticket to go pick it up.

Just to be clear... It was a prize, not a fine right?

The lesson is that in a win/not win contest, you want to do something no one else is doing. Maybe things go your way or maybe they don't. But you avoid the risk of being narrowly beat.

3rd place in our class was a student who dropped out two days into the stock game and never logged in.

short term trade is gambling unless you really have data and speed that you know how to use. Most trader uses charts and whats hot, the profits then runs on the greater fool theory.

You can go long and sell if there is a significant breakout. Then you hunt down the next value stock. A return is a return.

If you got into Nvidia ten years ago you would have to be dumb not to pull it now. There is market timing foolery, and then there is just being realistic.

> At the time we had a family friend that was working for a company about to announce a stock split along with a very good earnings report. He told me when to go all in on that stock, and i did exactly that. The day after that split my portfolio had more than doubled, beating the class record by a significant margin. Said record stood until the teacher retired.

That family friend seems like a complete idiot for passing on that information.

Plus a stock split on its own has no change to the value of a stock. There’s no reason for it to double overnight. In fact the direct effect on the price is the opposite as you have twice (or K times) as many shares and each is worth half (or 1/K). So the net effect is zero.

In theory you are right that stock splits have no bearing on returns, but in practice it is well documented (see e.g. [1]) that "stock splits and reverse splits often result in short-term abnormal returns even though such split events do not change any fundamental factors affecting the valuation of a firm's stock."

[1] https://www.sciencedirect.com/science/article/pii/S016726812...

> We provide evidence that the incomplete adjustment of share prices to splits or reverse splits can be attributed to heterogeneity in traders' cognitive abilities.

What a colorful turn of phrase.

The split was followed by the stock very quickly regaining it's previous per share value over the course of the trading day, though maybe it was over the next week; this was a few decades ago.

And i agree that he shouldn't have told me. A few years later i actually told him something along those lines. I appreciated it but he exposed himself to a lot of legal risk.

> Not sure if it was the lesson he meant to impart though since I think most took away from it that to win you need to lie, cheat and steal.

Better to teach reality than ideology, assuming you want to be a practitioner instead of a theorist.

Nothing ever bad happened when societies trade high trust for low trust. No sir. All roses and sunshine. Certainly worthwhile for me to bend or break any rule keeping me from my best life.

Hating cheaters does nothing to change the reality of it being an effective method for getting ahead.

Only in narrow circumstances and only to a point. Or at least that's my (limited) understanding of game theory.

You're a caricature of my original point and you don't even realize it.

IME the reality is that trust erodes as cheating becomes widespread. The consequences of less trust are significantly higher costs and more stress and fear.

FWIW, I don't advocate blind trust.

Perhaps you meant to say we should teach the reality that cheating exists and is bad; not to pretend it doesn't exist? Or that it's hopeless to be honest and trustworthy because some others may not be? Which leads to ... apathy or more cheating and less trust.

I never said cheating is moral. I said cheating is advantageous. You think cheaters care that they're hurting society? You can't be a cheater without being selfish.

It would if said cheaters were socially ostracized. Instead we make big budget movies about them, they get famous, and then earn money from the newfound attention.

And who’s fault is that exactky

Presumably people that break the rules in a high-trust society, or at least those that enable the rule-breaking.

So…. everyone?

Do you have a particular answer in mind, or a point to your question? Yes, societal rules in some sense depend on everyone enforcing them. But there are also people/organizations/etc. with more power to fund or support these violations of social rules, so presumably they’re more at fault too.

So your point is that if we fundamentally change society, ignoring the fact that this is integral to human nature itself, then we can fix this problem? That's about as helpful as saying we can have peace in the Middle East if we all just got along. It's comical how disconnected from reality it is, and yet you seem to think people should take it seriously.

No, I’m saying that people have stopped enforcing social rules and therefore cheating is increasingly a valid way to succeed. That’s what the point of the thread was about. Lots of things are fundamental to human nature; society is the process of controlling and channeling them.

Fair enough.

The whole rhetoric of high trust society sounds like a fallacious argument from consequences against well deserved distrust by claiming it will basically lead to the collapse of society.

I like Margin Call's quote - "be first, be smarter, or cheat".

Is what he did there illegal?

> hedge funds have historically outperformed the markets

According to Investopedia:

"From January 1994 to June 2023—through both bull and bear markets—the passive S&P 500 Index outperformed every major hedge fund strategy by over 2.8 percentage points in annualized return."

https://www.investopedia.com/articles/03/121003.asp

Good article. The grandparent comment is about hedge funds that outperform. Rather than offering the automatic diversification most people seek from hedge funds, his/her point is that some which are highly correlated to the market are laundering insider info.

Not just hedge funds.

Congressional democrats, and i am sure republicans too can outperform S&P 500

>" An exchange-traded fund (ETF) that tracks the stock trades of Democratic members of Congress has been outperforming the S&P 500 since its launch in 2023. "

https://markets.businessinsider.com/news/etf/etf-named-after...

Hillary Clinton cattle futures controversy - Wikipedia https://en.wikipedia.org/wiki/Hillary_Clinton_cattle_futures...

ETF Named After Nancy Pelosi, Tracking Congressional Democrats' Stock Trades, Surpasses S&P 500 with Tech Triumph | Markets Insider https://markets.businessinsider.com/news/etf/etf-named-after...

Sssh, we don't speak of the chicken man's chicken monies.

There is the fun example of a trade that was made before that information should have been physically able to arrive, based on speed-of-light limits. [1] (2013)

The article says "Presumably there will be a hard look into what exactly happened..." but I wonder how hard that look was, and how often that stuff still happens.

[1] https://www.washingtonpost.com/news/wonk/wp/2013/09/24/trade...

The regulators looking at gold futures would have been contending with manipulators placing massive orders in bad faith, trying to trigger stop loss. That all led up to the flash crash.

One thing I rarely hear people talk about is how many libertarians believe insider trading shouldn't be a crime. There are lots of libertarians in the financial industry, so there are likely lots of people in the financial industry who believe that laws against insider trading are evil.

So when I picture insider traders, I don't picture shady organized criminals doing things they know are bad. I picture cowbows believing themselves to be the good guys for freedomizing the market.

And when you think about all the people who believe that insider trading is a positive good, you kind of have to conclude that it's rampant. It's financially lucrative, hard to detect, harder to prove, relatively easy to hide, relatively easy to pin on someone below you in the org, and people think they're good for doing it. What force is keeping it in check?

Different POV: Insider trading should be legal because is fundamentally impossible to police fairly or effectively. The one caveat is ALL trades must be public in real-time (and not via shell corps, but showing the real beneficial owner). This way, insider trading can do public good by providing good market signals. Transparency would also reduce the advantages of insider trading.

Someone can correct me if I'm wrong (which I probably am), but my understanding is that legally the harm of insider trading is to the shareholders not to the fairness of the market.

So if I have insider knowledge of some earnings at BigCo and I trade on that, I've breached my fiduciary duty and in some sense stolen that info from BigCo.

I don't see how your scheme would address that.

Or, less abstractly, if you're preparing the earnings announcement for BigCo and you trade knowing earnings are bad, then you've leaked the announcement. And that gets worse, not better, if all trades are public and real time.

So I think total transparency in trading and insider trading interact in non-obvious ways.

> harm of insider trading is to the shareholders not to the fairness of the market.

no, insider trading doesn't harm the shareholders, except the ones who sold (or bought) without using said information (compared to someone who did have it).

The harm is indeed to the market - information assymetry means the other market participants, like the above shareholder, is not buying/selling as "correctly" as the ones doing insider trading.

This also leads to mis-pricing - something that decreases market efficiency.

But being so difficult to enforce, insider trading can't be fixed tbh. The best we can do, imho, is to make the signal go faster (which is what transparency aims to do). By making the signal go faster, insiders actually have very little time to actually "inside trade".

> leaked the announcement

the market _should_ know the earnings are bad. In fact, the market _should_ be making a prediction about the earnings in the aggregate. The information from an insider trader, if it were fully transparent, means that a company's shares will accurately reflect their earnings even if they didnt annouce it, and this makes the market more efficient.

If the market had all information then there would be no trading as the price would be correct and nobody would want to buy or sell, as doing so would be money-losing.

The market works because of different information, opinions, ideas that are available to different participants.

Trading on insider information is like doing a pump and dump, and should be illegal.

> there would be no trading as the price would be correct

the trading would happen when your personal risk is different from another trader. Future events (that have not yet happened) will also make each individual trader do trading based on their predictions.

It's absolutely not true that there's not going to be any trading. After all, unless every trader's internal risk rating and funding are _exactly_ the same, trading must happen, especially if information is very transparent.

> should be illegal.

just because it's declared illegal, doesn't mean it doesn't happen, nor people don't get away with it. It's why i claim that the next best thing is to _make_ it legal, but force the trade to be revealed instantly rather than have a 1 month time gap.

In the event that an insider (or potential insider) starts making large trades, there will be people observing and making similar trades, and thus the insider information (despite being obscured) is transmitted out via this trade signal. The faster this signal gets transmitted, the less insiders will have an opportunity to profit unfairly.

>>nobody would want to buy or sell, as doing so would be money-losing.

Somewhat related, there is a saying that caused me great hesitation for many years: "Remember, whatever trade you make, someone else is making the exact opposite bet; what is the likelihood which of you is wrong?". Now, in large, this hesitation is largely good, but not to excess.

I then realized that on the exact same trade people can have very different legitimate perspectives that do NOT invalidate yours, i.e., many situations where you can both be right, for your goals. E.g., a trader may have a great reason to sell a stock this minute while a long-term investor has an equally great reason to buy and accumulate the stock. Or, stocks can go in/out of specific investing criteria such as for growth, value, momentum, etc., and different portfolio managers will be selling and buying the same stock at the same minute and both be completely correct about the stock meeting their goals.

Noooo. The presumption of fairness in the market makes the market… without trust, you’ll have less capital, which hurts the shareholders.

My understanding was that insider trading is illegal because you are stealing (ideas, plans, news) from the company.

That’s pretty much the same as making it illegal. It’s a system of honest graft that requires for work and enforcement.

The public metadata would be immensely valuable, and nobody would want to comply. You’d be prosecuting people for concealing ownership.

>>You’d be prosecuting people for concealing ownership.

Yes, and concealing ownership over the long term is more difficult (in many cases, impossible, e.g., for executives with stock packages — exactly the set of most likely insider traders) and investigating and prosecuting it is far easier vs insider trading.

Seems like a win.

Also pretty easy to hide in the noise of a trading algorithm. Eg. Make bets that hold a collection of biotech firms on the day that the fda approves their treatment.

You could also up the exposure by choosing to hold N small market segments, that all overlap with the stock that’s being insider traded.

Like which ones do you suspect have done this?

Many funds do not use these sorts of strategies and the ones that do almost always underperform after fees.

So in aggregate there isn’t much evidence of outperformance.

That’s what some guys I knew in New York told me. People talk to each other and the hedge fund guys don’t really compete with each other.

My guess is they also make use of “expert networks” to access confidential information.

Anyone can pay for that. The problem is being able to read and digest it all.

I would take the idea further, that profitable retail traders are profitable, in large part, because corruption exists.

Not that they are engaged in corruption, but that any patterns they find primarily exist only as a result of the corruption of others.

It’s a working hypothesis at least.

I think it's the market makers and algorithmic traders. Not sure if they are corrupt. When things get famous the retail traders come in and the algos drop off and then the market makers start futzing over the arbitrage.

in the form of nudge nudge wink wink

Reply on news.ycombinator.com