What if ChatGPT was trained on decades of financial news and data?
niemanlab.org
niemanlab.org
e.g For it to be impactful in finance, being trained is not enough. It will need to take tons of new data in.
I wouldn't bet on it.
There's an entire episode on this.
- Growth's pretty impressive, turns out I can procrastinate about 1000% more efficiently
Here is the actual research paper from ,
https://arxiv.org/pdf/2303.17564.pdf
I can't tell if this is Bloomberg's attempt at solving a problem or if they are doing this out of FOMO.
It's really still too early to tell.
And so I can't help but wonder if this would outperform the market significantly due to seeing more patterns and holding more information than any individual trader or even whole desk ever could... OR if its total lack of any actual reasoning/deductive capabilities makes this just a total non-starter here. Or maybe it might be useful for day trading but not any kind of longer-term investment?
Why? It's famously bad at math after all. It can spit out quarterly reports and industry research back (so if analyst says "buy", ChagGPT will parrot that), but other than that it would - I think - work as a glorified sentiment analyser. Of course I may be wrong.
And we also need to remember that AI is used heavily in finances (and the best cutting edge methods are probably not published by the companies using them, since this is a zero sum game).
Customized neural networks for stock prediction have been around for decades, and for obvious reasons its been a domain that’s attracted a lot of attention and energy. LLMs obviously have utility in automatically processing news and the narrative portion of reports, but I don’t know that they are particularly likely to be useful as the main prediction engine rather than a tool to process text into inputs for specialized trading models (I’d be surprised if there hasn’t already been AI news sentiment analysis, etc., being used to supply input for these models, even before ChatGPT was available.)
Any edges, though, in that field get eaten up quickly in the arms race.
It has been done many times. It turns you predicting the flip of a coin can be done with extremely high levels of accuracy using traditional methods.
Rio Tinto et al have been warehousing data from the get go (circa Roman Empire with the oginal mine in Spain).
That's not the same as indexing which is a way to tune an existing model for a particular context.
Also, my understanding of LLMs (tiny) means they will only give you the most probable next sequence of words to your prompt based on the words in the training set. Has an LLM ever produced output that wasn’t buried in the training set somewhere? I.e. a novel answer to a novel question?
It would benefit from access to current news, but not continuous training (though more training is better).
> Also, my understanding of LLMs (tiny) means they will only give you the most probable next sequence of words to your prompt based on the words in the training set. Has an LLM ever produced output that wasn’t buried in the training set somewhere?
This is a misunderstanding. Its a pattern recognizing predictor, not a Markov chain, its not reproducing its training set, its predicting what a plausible document would look like from the universe from which the training set was sampled would look like if it started with the prompt, and the conpletion cna be vert different from anything in the training set if the prompt is, and (depending on tuning like “temperature”) potentially even if the prompt is exactly represented.
As people have pointed out, but I think should be repeated at every opportunity, this is the definition of bullshit. Superhumanly optimal bullshit.
Which has its uses, yes, but it is what it is. Bullshit is not the totality of human existence.
I feel like every time people equate this sort of thing with human consciousness, it is a way of saying "yes, bullshit is the totality of human existence".
another fun fact from the paper, it trained for 1.3m GPU hours and didn't get through the whole corpus, did 80% of 1 epoch (p. 16).
another fun fact, according to Twitter rumors GPT-5 is already training on 25,000 A100s w/8 GPUs each, so 4.8m GPU/hours per day.
it's a good paper with a lot more detail than OpenAI is releasing. also confirms OpenAI is quite a bit ahead. They compare with benchmarks they ran with Bloom and open source GPT-NeoX which they were able to run on same data, and reported benchmarks for GPT-3 since you can't run it yourself (well I guess you could sample the API), and BloombergGPT was competitive with GPT-3 which is a couple of years old now.
my takeaway was, this stuff is hard and OpenAI is crushing it.
I wonder if they will just use it for terminal features, or make the API available, and at what cost, and if they will license the model and weights for people who want to run it internally and maybe train incrementally on their own data. (not sure how feasible that is)
If true that is incredible. From what I understand there isn’t a function that relates the increase of training time/effort to performance. For example, twice the training doesn’t mean twice the performance. It could mean no increase or 1000x increase, no one really knows. Does anyone have any idea what to expect in terms of capabilities when this training is finished?
the bloomberg paper does talk about how they sized the model based on their compute budget and research on time to train ... megatron has 1t parameters
i'm intrigued by some of the knowledge graph performance benchmarks and wonder about training it alongside an explicit knowledge graph, instead of building its own implicit knowledge graph, despite the 'bitter lesson' about just letting simple models and massive computation do their thing.
It will be like you are at the horse race, a punter reads the horses form in the local paper. Versus the computer runs a tuned up linear regression on thousands of variables about past horse races, weather, tracks, horse injuries and so on going back 10 years.
What I replied to was:
> It'll do great. As everyone in the industry knows, past performance is always indicative of future results.
Assuming sarcasm, you are saying:
"It may not do great, because past performance is not always indicative of future results."
This is an adage to warn the everyday person that, say, while MSFT has produced great returns in the last 10 years, it does not guarantee it will carry on producing the same returns.
Which is fine.
But a sophisticated operation running machine learning on large live datasets to predict stock market movements doesn't need this adage. And in addition, it is possible for them to beat the market if they can make predications and connections that no one else is making.
Understand that is not "they can't lose any bet". It is more "they will make money over all bets on average that gives a higher return than the market".
Open question as to whether the generative model can do this by turning on a switch, or whether it is tuned and modified by experts to understand something about current affairs.
https://www.guinnessworldrecords.com/world-records/most-succ...
[0] https://en.wikipedia.org/wiki/A_Random_Walk_Down_Wall_Street
consider a stock picker who starts with 256 people, tells 128 of them that next week Tesla will go up and Twitter will go down. Whichever happens, take that 128 strong cohort, tell half of them that the next week, Ford goes up and Chrysler goes down.
Etc.
repeat for 8 weeks.
At the end, the stock picker has a true believer: someone who thinks they saw the stock picker successfully "know" for 8 weeks in a row which stocks would go up.
Change the numbers around a bit, don't run until the true believer pool size is one, and you've got ... the real world.
If we could really attribute this task to randomness wouldn't this save a lof of money to financial institutions considering how much the same people earn for these tasks?
Erm, Kahnemann got a Nobel price in economy for his work with Amos Tversky over a decade which this was a part of. I am not sure what you need here, do you need a list of papers by Kahnemann and Tversky? With Terry Odean thrown in for good measure? I am not sure who the "we" are here who do not see these papers but they do exist.
The actual conclusion is that most financial managers are fakes, and laypersons cannot tell the legit from the fakes.
A group of men in suits would ask me, oh Oracle, what should I invest in? And I'd tell them "My intuition tells me.. buy ACME stocks". Another group would show up and ask the same thing, and I'd answer the same, and the first group of men would make a profit, and huzzah, the oracle is right again!
But of course I'd tell a good friend to buy ACME stock before all of them did, or maybe this friend would be the one telling me which stock I should recommend to the people...
So.. the opposite of Jim Cramer.
https://uploads-ssl.webflow.com/637240a49ba56f7bfbe82c84/640...
I mean, "do the opposite of a stupid thing to succeed" is traditionally funny because it obviously doesn't work - because there are generally infinite ways to fail...
You tell everybody you're a brilliant stock picker, and you get, let's say, 256 people to follow your advice.
So you choose a stock, and tell half of them it'll go up and half of them it'll go down.
That means 128 of your followers are guaranteed to think you predicted correctly.
So you do it again, and again, until you get down to like one or two people who think you are a genius.
Then you leverage your true believer(s) faith in you.
Still, BloombergGPT has knowledge far in excess of a monkey, and if it's anything like ChatGPT, is able to provide reasoning as to how it arrived that that conclusion. Time will tell if, granted a brokerage account, how it will perform relative the market. It may or may not outperform a monkey throwing darts.