TradingGPT: LLM multi-agent framework for enhanced financial trading performance | Hacker News Reader

TradingGPT: LLM multi-agent framework for enhanced financial trading performance

arxiv.org

29 pointsjasondavies15 comments

Edit: Someone pointed out this might be a VC trap, which would explain why there's such breathless writing about a bogus model with no actual results included.

This is a whopper of ai-will-totally-take-over-trading nonsense paper, you'll become less informed about reality if you read it. I'm not going to cover everything but to make sure nobody thinks some new gpt is going to give trading recommendations:

* It's not clear the group ever trained a model. If they have, there's no data about that. There's an infinitude of subtle traps when training financial models you have to be aware of.

* The proposed training and evaluation periods are remarkably short for the holding periods they suggest, if they were to have included good test results

* There's no information about how the exact timing of the data feeds they're giving, how they measure the price+time+cost of execution, how they think about market impact, etc.

* There's no mention of risk management aside from some vague risk-preference ideas the gpt might theoretically have

Putting that aside, there's a fundamental misconception held by the authors. If you have some mega-network that can parse all sorts of financial information/statements/whatever and meaningfully tell you information about the future, you're not going to add a ton of nonsense about understanding written language prompts to have a discussion with the user. The actually valuable thing is the predicted forward returns / target portfolio / whatever piece of information you're trying to get.

Renaissance is the benchmark for AI/ML applied to trading. Naturally, they are completely secret.

Euclidean, the fund that manages the personal wealth of some of Renaissance's key personnel, also uses AI/ML to trading... actually, to investing, more long-term.

They have publicly shared interesting research findings worth reading:

https://www.euclidean.com/data-posts-machine-learning

"Don't assume an investment will continue to do well in the future simply because it's done well in the past."

Statistical arbitrage trades don't tend to be like "yolo long these great stocks today" unless you have access to information very few other players do (and even then you try and only gain exposure to the factors you are informed about). They tend to trade complex relationships between many assets in a universe, like if your universe of assets has moved in a way that's out-of-line with how the assets tend to move together.

Where’s the beef? There’s no data. No evaluation of how well the model does. They’re also going to limit themselves to just the pandemic period (which is probably a very biased time period in the stock market).

As with any research / publication around trading and the stock market: if this strategy worked and provided an advantage, it would not be published free for all to read.

The one interesting thing here is using three different sets of agents to handle information at three levels of scope/timeframe. From my limited experience with trading, there are predictable behaviour patterns (not the information itself, but how people act on it), and these patterns tend to be self-similar across magnitudes of scale (not unlike fractals).

Otherwise, this is an unusually short paper, and light on data analysis / evidence.

I feel like 90% of <something>GPT projects are VC traps. Why would anyone discard models that are specifically designed for financial trading and use transformers that are not suitable for the job?

One thing I always assume about quantitative finance research, is that by the time it’s published in a journal some trading desk on Wall Street has already been using some version of the result for years. There is simply too much money to be made. Unfortunately for us there is also incredible secrecy and siloing of these innovative results.

I think "useful but late" is a tiny minority of the research that might come across your feed. The more common category is "advertising formatted as research".

There are many traditional money managers that would like to allocate to quant strategies because they are ostensibly uncorrelated with their existing investments. But the exact fact that these investors are traditional makes it difficult for them to access and evaluate meaningful quant opportunities. As a result there is a huge volume of aspirational research designed to attract attention and thus funding from this group, but which hasn't had any meaningful peer or market vetting.

There is literally no content in this paper besides a rough idea of how something like this could work, with a metric ton of bullshit bingo mixed in there. Man I really hate the academic paper system.

Maybe it works great and is super advanced, but my experiences so far with ChatGPT (even 4) shows that it tends to make basic mathematical errors sometimes.

You'll excuse me if I am skeptical that an LLM is going to do a better job at figuring out the market then some of the most highly paid minds in the world who have been at the problem for decades now.

monkeys throwing darts have often outperformed the index. what does that tell you ? evaluating trading algorithms for sustainable edge is science.

the competitive and zero sum nature of trading requires a deeper and almost scientific bent of mind to think well.

Big Knight Capital vibes...

That was a code management f up though. Someone plugged a test battery into production IIRC.

Wow. This is the future. I am going to launch a product based on this

Reply on news.ycombinator.com