Hacker News Reader

Comment by tubby12345 | Hacker News Reader

>topological quantum field theory

lolol. i will bet all of my 401k that they absolutely do not.

i've interviewed at a couple of hedge funds and have lots of friends at various prop shops/hft firms/etc. most of them are using linear models. the ones that have market maker businesses deploy those linear models to fpgas.

people act like these places are spooky magic cauldrons of physics + math + computer science. they're not (but it's definitely in their interest to promote such rumors). what they do have is very very robust data collection/aggregation infrastructure and backtesting systems. the rest is just correctness of execution of strats.

View on HN

I'll second that. I've done options trading, trend following, and HFT. Often you need smart people to do simple things, but they are still simple things. Gotta remember there's a lot of noise in financial data, and you don't actually have as much of it as people think. Even all the ticks on all the exchanges fit in a few GB of binary per day.

RenTech is especially good at acting like they're from another planet, which I'm sure helps them attract those stellar mathematicians.

Your especially right about the noise in financial data. Nice thing about linear models (vs i.e. random forest, or even worse deep learning) is that they are limited in how they can overfit the noise. They only fit a known function to known regressors.

And, even more important, they aren't susceptible to adversarial activity like a deep net could be.

That's actually been surprising to me but makes sense: "the entire history of the stock market" is not that big in terms of data, and the dynamics change frequently enough to where things that happened in 2000 are almost useless from a machine learning standpoint, my models perform best when trained on a much more selective subset (I believe because the reasons for trades and market participants keep changing, so the dynamics change enough to where old rules don't apply anymore).

Correct. You can make money using only yesterday’s data for training purposes.

Can you distinguish noise from non-linear correlations? I feel like you can never prove the absence of non-linear correlation, but only showing it‘s presence once modelled/detected

RenTech is also basically the only largest hedge fund that has beat the market by a wide margin for decades. An average 66% annual return in their Medallion Fund since 1988 according to wikipedia. That is pretty much insane and seems to be something only space aliens could do. If anyone is using quantum computers to help trade, it is probably them.

Worth noting that their public funds don’t return anything like the rumored Medallion fund returns (though they are generally pretty good).

Medallion is also kept very small because I would guess the tricks that they use to keep those returns so high (maybe even some sort of topological quantum field theory!) don't scale well to larger investments.

Not sure if I would call ~$10 billion very small and making $6-7 billion per year is good chunk of money. But, yea, I'm sure that would get bigger if they thought they could keep their returns high.

The Medallion Fund has already reached their max investment amount so while it's ground-breaking, no new RenTech employees can take advantage of it.

> An average 66% annual return in their Medallion Fund since 1988 according to wikipedia.

The claim, the secondary (tertiary?) source, and the primary source make that all hard to not smile at.

66% annualized returns are more likely to be pyramid schemes - but that couldn't happen, could it?

That's because it's not 66% but instead $6B. If they had twice as much capital it would be 33%, and if they had ten times as much capital it would be 6.6%. I'm simplifying to show the main point.

> Even all the ticks on all the exchanges fit in a few GB of binary per day.

This isn’t true once you include equity options, which have several orders of magnitude more data.

However your general point still stands, because most options trading strategies don’t need such extreme granularity of data. Much of it can be ignored, or close to it.

You're right overall that most of these places are a lot less flashy on the inside (have worked at a couple). Linear models everywhere as you said

However I think you're underestimating RenTech here. They're genuinely just in another league compared to what most people consider the "elite" quant shops. You're not getting in unless you're an actually impressive academic with a track-record, so I wouldn't be surprised if they're trying some weird stuff that other shops can't even understand (although I would imagine in small size vs. more vanilla stuff).

IMO places like JS/CitSec do a really good job of bombarding campuses to inculcate this idea they're the absolute apex of mathematical wizardry, but the places that are really doing some dark magic shit aren't trying to get undergrads to apply to them. Ofc maybe I'm falling for the same kind of propaganda for RenTech

All of this wizardry operates under the premise that given just the right meta-meta-meta-formulae some tiny edge over randomness can be squeezed from the mountain of historical data by sheer willpower and brute force computation. It cannot. The sooner people accept this as an iron rule, the sooner they'll stop falling for scams that promise to foretell the future.

But it obviously can. The returns of Renaissance’s Medallion fund cannot be explained by randomness. If every company in America was a hedge fund since 1776, the returns of Medallion would not arise by chance. They clearly have an edge (and don’t even accept outsider money, so no need to falsify).

I think your definitely idolizing them a bit too much. Maybe a bit ahead of the curve but not doing anything super fancy (although I guess those two statements are somewhat contradictory).

Just from snippets I've sort of heard/read about, I think they were one of the earlier ones to move into HFT (although maybe not the super fast infrastructure heavy type these days). In some interview Simmons said they realized returns became more predictable they shorter the time frame they looked at and they pushed it to the extreme. I think there is also reason to suspect that they may have adopted some NLP strategies early on as well since Mercer was involved in that or something, and they initially hired a bulk of their team from IBMs NLP research team. Also they did not dodge 2008 completely, in some interview they said they lost close to / more than half their portfolio value in the market crash, but because they didn't have stupid leverage or outside investors or something like that, and also because they trusted their models, they didn't sell and held on. So maybe just slightly better execution but mostly the same.

Anyways, I was reading about these guys back in 09 when quant trading wasn't so blown up. Now every kid whose decent at math seems to want to be the next renaissance, which just makes me feel like the best years for that are over.

I’m not idolising them, I’m just framing their returns in the correct perspective. The probability of any of the top firms existing by chance is astronomically small, that’s all I’m saying. Same is true of BlueCrest etc.

As far as I know the NLP stuff is more to do with similar techniques being applied to market data, rather than actual speech recognition or whatever. Hidden Markov models and the like.

fancy or not an edge is an edge.

I’m not so sure the best years are over. The passive investing trend might work to the contrary and make the markets less efficient

They are well known for insider trading and other market abuses. Of course they'd rather everyone believed it was all the PhDs they've hired, but it's just a smokescreen.

The SEC will pay you a lot of money if you can substantiate those claims.

https://www.cityam.com/renaissance-hedge-fund-pay-5bn-back-t...

That is one example.

That’s neither insider trading nor market abuse. They did something in a grey area regarding taxes, then decided to pay the bill rather than fighting in court to determine if it was or wasn’t legal.

They are the best at extracting information from publicly available data at fast speed, no more than that.

If you hire very intelligent people to do just that, you are doing it right. But the info is out there for everybody to see. They just arrive earlier than others. How? That is the secret.

Medallion distributes their earnings and stays a fixed size rather than compounding, so it's a category error to compare their returns to most hedge funds. (At 66% return for 30 years, they'd own everything in the world otherwise). They're more like an internal prop-trading firm, which makes their returns good but not insane.

I’m well aware it’s not compounded, it’s still a worthwhile comparison. We’re comparing ability to capture alpha, they’re clearly among the best in the world at that. 66% is insane for 30 years as a prop desk even with a fixed capacity, what makes you say it’s just good? Who is doing better?

Clarification: the data appears random to you.

>However I think you're underestimating RenTech here. They're genuinely just in another league compared to what most people consider the "elite" quant shops.

People said the same thing about LTCM. They had multiple Nobel Prize winners on staff, literally the people who wrote the Economics and Finance textbooks.

Then a decade later, history repeated with all of the prop trading outfits doing securitization.

It's always the same ingredients: 1) a theoretically sound strategy for taking advantage of some arbitrage opportunity 2) the assumption that positions can actually be liquidated on demand at the prevailing price and 3) enormous amounts of leverage. Then something happens that wasn't accounted for (e.g., sovereign default, counterparty default, etc.) and suddenly the strategy is no longer sound ("in a crisis all correlations go to one"), the liquidity assumption is no longer true ("where are all the buyers?"), and the leverage puts you out of business ("the market can stay irrational longer than you can stay solvent"). People never learn.

RenTech was founded years before LTCM and is still going strong. It's not repetition of history, they've outlasted or are older than most of the trading desks ever. They're basically a business providing a service to other market participants and executing that incredibly well, it's quite unlike the arbitrage LTCM was involved with.

And Bernie Madoff founded his firm two decades before Ren Tech, hell at one point he was the Chairman of NASDAQ.

That was a scam. Are you accusing RenTech of being dangerously overextended like LTCM, or a scam like Madoff? Considering the Medallion fund takes no outsider capital, that would be quite a strange way to run a Ponzi scheme.

Seconded. I think generally people give way too little weight to actually doing things well. There can be some clever mathematical ideas in there here and there, but in the end it has to come together into a running organization that makes little mistakes and can run consistently (while also observing a lot regulations).

For me the "spooky magic" is more the ability to pull this off in size.

Edit: I do have friends on the buy side that use pretty fancy models, but those places are not in the HFT/market making game. But using those models does not take away from needing to be able to translate the edge consistently and efficiently.

Where are Reinforcement Learning algorithms standing today in Finance? I remember ~3 years ago they were supposed to be the next big thing.

This sounds a lot more plausible. Being able to extract signals faster or that others cannot seems a much more likely strategy.

Linear models, so not even deep learning models?

No. Most "advanced" ML methods are extremely difficult to get to work in finance. You have a limited dataset that is low signal to noise ratio, highly non-normal and heteroscedastic. Crucially, you can't easily make more data (and no, synthetizing new price data is not an option in most cases). This makes the upfront costs very high, and you then have to prove that you can beat the performance of simple models, and not just by a little. A complicated ML model that offers a 10% better risk/reward ratio (e.g. 2.2 Sharpe instead of 2) is a complete failure, because you traded a simple model that is easy to reason about intuitively for a total blackbox that is very difficult to understand.

Sure, you've got the quintessential marketing induced ML overlay that many firms do, but in all cases I've seen so far it's completely defanged and really there only so that it can serve as a marketing move.

Really depends on time horizon you're talking about. They actually work pretty well for HFT, there's plenty of data around, and most of the information is just in market data - no nasty low frequency stuff to deal with like news, earnings, alternative data, insider trading, butterflies flapping wings in china etc. But the problem is by the time your GPU spits out a datapoint somebody else can go in and trade a few thousand times in the meantime. State of the art on the most heavy competed exchanges is that your fpga (or even asic) with a fiber connected directly to the exchange needs to start sending ethernet/ip headers even before it made up its mind what it wants to send in the payload.

At lower frequencies when the data gets thin and noise/overfitting is a major problem, yeah it makes sense to use simpler models. Bias/variance tradeoff in action.

Fair points. I talked with a couple of HFTs and the general view was that their asymptotic backtests look promising on GPUs but that for most of the liquid markets the latency is way too high - basically confirming what you write.

On niche markets with low liquidity, one doesn't have such tight latency envelopes but those markets also offer more opportunities in general so again there's no real justification to use fancy ML models or GPUs in general.

GPUs are too slow for HFT trading, yes. Deep neural networks in general do work in this domain, people do use them, but the stuff that you can profitably deploy into production is not going to be your vanilla garden variety neural networks, there's a lot of extra engineering required to make it fast enough.

i interviewed at DE Shaw not less than two weeks ago. my research area is GPUs and etc. they don't have a GPUs team. i passed on the offer.

Why not both? Linear models for absolute lowest tick-to-trade latency doesn't preclude you from using fancier stuff at earlier modelling steps. Final linear models you ship to fpga can be mere distillations/triggers

I mean, every model's output is in the eye of the beholder (or modeler). Take any unpredictable wave-like chart and ask someone which direction it's going in next - and guarantee you'll throw $100M on their choice. Hey! You just successfully collapsed a wave! It's funny how after all these millennia, most people still don't understand that someone claiming to divine the future from bloody egg yolks or whatever is just manipulating the future by getting people to believe in their magical divination.

Reply on news.ycombinator.com