Financial Statement Analysis with Large Language Models
papers.ssrn.com
papers.ssrn.com
This figure compares the prediction performance of GPT and quantitative models based on machine learning. Stepwise Logistic follows Ou and Penman (1989)’s structure with their 59 financial predictors. ANN is a three-layer artificial neural network model using the same set of variables as in Ou and Penman (1989). GPT (with CoT) provides the model with financial statement information and detailed chain-of-thought prompts. We report average accuracy (the percentage of correct predictions out of total predictions) for each method (left) and F1 score (right). We obtain bootstrapped standard errors by randomly sampling 1,000 observations 1,000 times and include 95% confidence intervals.People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.
Not having made it big myself I obviously don’t know the meta these days, but last I had any inside baseball, the non-stationarity and friction just kill you on trying to get fancy as opposed to just nailing it on the fundamentals.
Extreme execution quality is a game, people make money in both traditional liquidity provision and agency execution by being fast as hell and managing risk well.
Individual signals that are individually somewhat mundane but composed well via straightforward linear-ish regressions is a game: people get (ever decaying) alpha out of bright ideas (and rotate new signals in).
And I’m sure that LLMs have started playing a role, there’s a legitimate capability increase in spite of the dubious production-worthiness.
But as a blind wager, I bet prop trading is about what it was 5 years ago on better gear: elite execution (no pun intended) on known-good ways to generate alpha.
1. Your automated system should be as fast as possible.
2. Stick with known, basic fundamental strategies.
3. Try new ideas around how to give those same strategies more predictive power (signal).
#1 is straight technical execution.
#3 is constantly evolving.
Is how I understood this.
And as sort of an afterthought I guess the better you are at #1 the less good you need to be at #3 and the worse you are at #1 the better you need to be at #3?
Mind elaborating?
Anyone who has figured out something relatively profitable isn't telling anyone how they did it.
Corollary: someone who is selling you tools or strategies on how to make tons and tons of money, is probably not making tons and tons of money employing said tools and strategies, but instead making their money by having you buy their advice.
The fact that you can't reveal how means you can't prove you're not Ponzi. If you reveal how, they don't need you.
This is why I am wary of all those +10 minute YT vids telling you how you can't make significant amounts of money quickly or reliably in a short amount of time with very limited capital.
If you were running this yourself with $1M input capital, that'd be $20k/year per 1M of input - so $20K is a nice number to try and beat selling a product that promulgates a strategy.
But you're going to run into the question from people using the product: "Yeah - but HOW DOES IT WORK??!!!" and once you tell them does your ability to get paid disappear? Do they simply re-package your strategy as their own and cease to pay you (and worse start charging for your work)? Is your strategy so complicated that the value of the tool itself doing the heavy lifting makes it sticky?
Getting people to put their money into some Black Box kind of strategy would probably be challenging - but Ive never tried it - it may be easier than giving away free beer for all I know. Sounds like a fun MVP effort really. Give it a try - who knows what might happen.
Maybe it's just what I know, but I can't help but think the "strategies" are a lot like security exploits--some cleverness, some technical facility, but mainly the result of staring at the system for a really long time and stumbling on things.
Because then your competition knows which strategies don't work, and also what types of strategies you work on.
Don't leak information.
And then everything regresses to the Dark Forest game theory.
I am assuming, he/she minds a lot.
Quant trading is about "going fast" or "being super right", so either you'd need to be sitting on some huge llama.cpp/transformer improvement (possible but unlikely) or its more likely just some boring math applied faster than others.
Even if they are using a "LLM", they wont tell you or even hint at it - "efficient market" n all that.
Remember all quants need to be "the smartest in the world" or their whole industry falls apart, wait till you find out its all "high school math" based on algo's largely derived 30/40 years ago (okay not as true for "quants" but most "trading" isn't as complex as they'd like you/us to believe).
Saying it's all high school math is a bit of a loaded phrase. "High school math" incorporates basically all practical computer science and machine learning and statistics.
If I suspect you could probably build a particle accelerator without using more math than a bit of calculus - that doesn't make it easy or simple to build one.
Very few people I've worked with have ever said they are doing cutting edge math - it's more like scientific research . The space of ideas is huge, and the ways to ruin yourself innumerable. It's more about people who have a scientific mindset who can make progress in a very high noise and adaptive environment.
It's probably more about avoiding blunders than it is having some genius paradigm shifting idea.
Moreover, the collaborative environment at a prop firm can't be understated. Ideas and strategies are continuously debated, tested, and refined. This collective brainpower often leads to more robust strategies than what you might come up with on your own.
That said, there are successful solo traders, but they often specialize in niche markets where they can leverage unique insights or strategies that aren't as capital intensive. It's definitely not for everyone and comes with its own set of challenges and risks.
A car designer still needs a car factory of some sort, and there's a negotiation there about how the winnings are divided.
In the trading world there are a variety of strategies. Something very infra dependent is not going to be easy to move to a new shop. But there are shops that will do a deal with you depending on what knowledge you are bringing, what infra they have, what your funding needs are, what data you need, and so on.
I too believe this is key towards successful trading. Put in other words, even with an exceptionally successful algorithm, you still need a really good system for managing capital.
In this line of business, your capital is the raw material. You cannot operate without money. A highly leveraged setup can get completely wiped out during massive swings - triggering margin calls and automatic liquidation of positions at the worst possible price (maximizing your loss). Just ask ex-billionaire investor/trader Bill Hwang[1].
1. https://www.bloomberg.com/news/features/2021-04-08/how-bill-...
Im responding to the comment "do use llama3" not "breakdown your start"
> Very few people I've worked with have ever said they are doing cutting edge math - it's more like scientific research . The space of ideas is huge, and the ways to ruin yourself innumerable. It's more about people who have a scientific mindset who can make progress in a very high noise and adaptive environment.
This statement is largely true of any "edge research", as I watch the loss totals flow by on my 3rd monitor I can think of 30 different avenues of exploration (of which none are related to finance).
Trading is largely high school Math, on top of very complex code, infrastructure, and optimizations.
I know nothing about this world, but with things like "doctor rediscovers integration" I can't help but wonder if it's not deception but ignorance - that they think it really is where math complexity tops out at.
It is neither deception or ignorance.
It's the same reason some of the best physics students get PhD studentships where they are basically doing linear regression on some data.
Being very good at most disciplines is about having the fundamentals absolutely nailed.
In chess for example, you will probably need to get to a reasonably high level before you will be sure to see players not making obvious blunders.
Why do tech firms want developers who can write bubble sort backward in assembly when they'll never do anything that fundamental in their career? Because to get to that level you have to (usually) build solid mastery of the stuff you will use.
Trading is truly a complex endeavour - anybody who says it isn't has never tried to do it from scratch.
Id say the industry average for somebody moving to a new firm and trying to replicate what they did at their old firm is about 5%.
Im not sure what you'd call a problem where somebody has seen an existing solution, worked for years on it and in the general domain, and still would only have a 5% chance of reproducing that solution.
> It is neither deception or ignorance.
How is it not ignorance of math?
> In chess for example, you will probably need to get to a reasonably high level before you will be sure to see players not making obvious blunders.
To extend the chess analogy, having the fundamentals absolutely nailed is critical at even a mid-level, because the payoff/effort ratio in avoiding blunders/mistakes is much higher than innovating or being creative.
The process of getting to a higher level involves rote learning of common tactics so you can instantly recognize opportunities, and then eventually learning deep into "opening theory" which is memorizing 10 starting moves + their replies because people much better than you have written lengthy books on the long-term ramifications of making certain moves. You're learning a vast repertoire of "existing solutions" so you can reproduce them on-demand, because those solutions are battle-tested to not have weaknesses.
Chess is a game where the amount you have to lose by being wrong is much higher than what you gain by being right. Fields where this is the case want to ensure to a greater extent that people focus on the fundamentals before they start coming up with new ideas.
you mean backporting a high-level implementation to assembly? Or is writing code "backward" some crazy challenge interviewees have to do now?
Because 95% of experienced candidates in trading were fired or are trying to scam their next employer.
“Oh, yeah, my <insert HFT pipeline or statarb model> can do sharpe <random int 1 to 10> for <random int 10 to 100> million pnl per year. Trust me bro”. Fucking annoying
Orders of magnitude more leave their jobs of their choosing than are fired.
These PMs are not the ones job hopping every year.
And 95% of interview candidates are not PMs.
> So the only scam is scamming yourself into a low salary position for a couple years till they fire you.
200k-300k USD salary is not low.
And 1 year garden leave / non compete? That’s literally 0.5M over 2 years for doing jack shit.
This is very appealing for tech SWEs or MBA product managers who are all talk and no walk.
But even with profit share / pnl cut, many firms pay you a salary, even before you turn a profit. It eventually gets deducted when you turn a profit.
> Orders of magnitude more leave their jobs of their choosing than are fired.
Hedge fund, maybe. Prop trading, no.
The engineers are are incredibly smart people, and so the bots are "incredibly smart" but "finance" is criticised by "true academics" because finance is where brains go to die.
To use popular science "the three body problem" is much harder than "arb trade $10M profitably for a nice life in NYC", you just get paid less for solving the former.
It's like math v engineering - you can come up with some beautiful pde theory to describe this column in a building will bend under dynamic load and use it to figure out exactly the proportions.
But engineering is about figuring out "just make its ratio of width to height greater than x"
Because the goal is different - it's not about coming up with the most pleasing description or finding the most accurate model of something. It's about making stuff in the real world in a practical, reliable way.
The three body problem is also harder than running experiments in the LHC or analysing Hubble data or treating sick kids or building roads or running a business.
Anybody who says that finance is where brains go to die might do well to look in the mirror at their own brain. There are difficult challenges for smart people in basically every industry - anybody suggesting that people not working in academia are in some way stupider should probably reconsider the quality of their own brain.
There are many many reasons to dislike finance. That it is somehow pedestrian or for the less clever people is not true. Nobody who espouses the points you've made has ever put their money where there mouth is. Why not start a firm, making a billion dollars a year because you're so smart and fund fusion research with it? Because it's obviously way more difficult than they make out.
Not that it's particularly relevant to this discussion but the three body problem is easy. You can solve it numerically on a laptop with insane precision (much more precisely than would be useful for anything) or also write down an analytic solution (which is ugly and useless because it converge s extremely slowly, but still. See wikipedia.org/wiki/Three-body_problem).
> Unlike the two-body problem, the three-body problem has no general closed-form solution,[1] and it is impossible to write a standard equation that gives the exact movements of three bodies orbiting each other in space.
This seems like the opposite of your claim.
A similar claim is that roots of polynomials of degree 5 (and over) have no "general closed form solution" (with, as usual, the implicit qualification: "in terms of functions I'm currently comfortable with because I've seen them a lot"). That doesn't mean it's a difficult problem.
The two problems have in common that they are significantly harder than their smaller versions (two bodies, or degree 4). Historically, people spent a lot of time trying to find solutions for the larger problems in terms of the same functions that can be used to solve the smaller problems (conic sections, radicals). That turned out to not be possible. This is the historical origin of the meme "three body problem is unsolvable".
For polynomial roots, see wikipedia.org/wiki/Elliptic_function.
My interpretation of "finance is where brains go to die" is more along the lines of finance being less good for society at large compared to pure science. Like if someone invents something new and useful in a lab for their phd, then they go find a job in finance. The brain died because it was onto something and then abandoned it for being a cog in the machine.
(Note that I personally have no opinion on this topic, as I'm not sufficiently informed to have one.)
The op is making some implication across numerous posts that it's all basically a big con and it's all very simple.
It is like claiming you don't need to be rocket scientist to go to the moon because they just use metal and screws.
The individual parts might be simple in isolation. But it is the complexity of conducting large scale, large scope research in an environment that gives you limited feedback and will adapt to your own behaviour changes that is where the smarts are needed.
OP seems to not understand the inherent difficult of doing any research.
Almost anybody could be taught to make a simple circuit and battery from some basic raw materials. The fact it is simple and easy now we know the answer does not mean it was simple or easy to discover. Some of the greatest minds dedicated their entire lives to discovering things that now most 10 years olds understand. That doesn't imply you only need to have the intellect of a 10 year old to make fundamental breakthroughs in science.
Working in quant trading is almost pure research - and so it requires a certain level of intellect - probably at least the intellect required to pursue a quantitative PhD successfully (not that they need the PhD but they need the capacity to be able to do one).
e.g. LMAX Disruptor was a pretty impressive concurrency library a decade ago:
https://diabetesjournals.org/care/article/17/2/152/17985/A-M...
What algos are you referring to derived 30 or 40 years ago? Do you understand the decay for a typical strategy? None of this makes any sense.
To be "super right" you just have to make money over a timeline, you set, according to your own models. If I choose a 5 year timeline for a portfolio, I just have to show my portfolio outperforming "your preferred index here" over that timeline - simple (kind of, I ignore other metrics than "make me money" here).
Depending on what your trading will depend on which algo's you will use, the way to calculate the price of an Option/Derivative hasn't changed in my understanding for 20/30 years - how fast you can calculate, forecast, and trade on that information has.
My statement wont hold true in a conversation with an "investing legend", but to the audiance who asks "do you use llama3" its clearly an appropriate response.
Aside from the "theoretical" developments the other comment mentioned, your implication that there is some fixed truth is not reflected in my career.
Anybody who has even a passing familiarity with doing quant research would understand that black scholes and it's descendants are very basic results about basic assumptions. It says if the price is certain types of random walk and also crucially a martingale and Markov - then there is a closed form answer.
First and foremost black scholes is inconsistent with the market it tries to describe (vol smiles anyone??), so anybody claiming it's how you should price options has never been anywhere near trading options in a way that doesn't shit money away.
In reality the assumptions don't hold - log returns aren't gaussian, the process is almost certainly neither Markov or martingale.
The guys doing the very best option pricing are building empirical (so not theoretical) models that adjust for all sorts stuff like temporary correlations that appear between assets, dynamics of how different instruments move together, autocorrelation in market behaviour spikes and patterns of irregular events and hundreds of other things .
I don't know of any firm anywhere that is trading profitably at scale and is using 20 year old or even purely theoretical models.
The entire industry moved away from the theory driven approach about 20 years ago for the simple reason that is inferior in every way to the data driven approach that now dominates
That’s not true. It is true that the black scholes model was found in the 70s but since then you have
- stochastic vol models
- jump diffusion
-local vol or Dupire models
- levy process
- binomial pricing models
all came well After the initial model was derived.
Also a lot of work in how to calculate vols or prices far faster has happened.
The industry has definitely changed a lot in the past 20 years.
Since the GFC it’s not about crazy new products (on derivatives desks), but it’s about getting discounting/funding rates precisely right (depending on counterparty, collateral and netting agreements, onshore/offshore, etc), and about compliance and reporting.
Not true. Most of the magic happens in estimating the volatility surface, BSM's magic variable. But I've also seen interesting work in expanding the rates components. All this before we get into the drift functions.
In vanilla equity options, sure. But that’s like saying we solved rockets in WWII. The foundational models were derived by then; everything that followed was refinement, extension and application.
How you can calculate fast, forecast, and trade on that information has
There. Fixed it for you. ;)
The old joke of two economists ignoring a possible $100 bill on the sidewalk is an ironic adage. There are hundreds of bills on the sidewalk, the real problem is prioritizing which bills to pick up before the 50mph steamroller blindsides those courageous enough to dare play.
It's a lot like quantum mechanics or whatever it is that makes the observation of a photon changes. Except with the caveat that the first to recognize the trend can direct it's change (for profit).
Going fast means scalping?
That's not to suggest that Renaissance is going to start using Chat GPT tomorrow, but maybe in a few years they'll be using fine tuned versions of LLMs in addition to whatever they're doing today.
Even if it's not going to compete with the state of the art models for something, a single model capable of many things is still useful, and demonstrating domains where they are applicable (if not state of the art) is still beneficial.
"In a few years" you'd have the benefit of the current, bespoke tools, plus all the work you've put into improving them in the meantime.
And the LLM would still be behind, unless you believe that at some point in the future, a radically better solution will simply emerge from the model.
That is, the bet is that at some point, magic emerges from the machine that renders all domain-specialist tooling irrelevant, and one or two general AI companies can hoover up all sorts of areas of specialism. And in the meantime, they get all the investment money.
Why is it that we wouldn't trust a generalist over a specialist in any walk of life, but in AI we expect one day to be able to?
I have a slightly more cynical take: Those LLMs are not actually general models, but niche specialists on correlated text-fragments.
This means human exuberance is riding on the (questionable) idea that a really good text-correlation specialist can effectively impersonate a general AI.
Even worse: Some people assume an exceptional text-specialist model will effectively meta-impersonate a generalist model impersonating a different kind of specialist!
Eloquently put :-)
If there were some super generalist that could then the specialist would have no power.
"I didn't violate a red light. I wasn't even driving, the AI was!"
"The AI said you did, that's 50,000 yuan please."
The specialist is a result of his general intelligence though.
So it all needs checking. It's the classic LLM situation. If you're trained enough to spot the errors, the analysis wouldn't take you much time in the first place. And if you're not trained enough to spot the errors...
And let's say it does work. It's like automated exchange betting robots. As soon as everyone has access to a robot that can exploit some hidden pattern in the data for a tiny marginal gain, the price changes and the gain collapses.
So if everyone has the same access to the same banal, general analysis tools, you know what's going to happen: the advantage disappears.
All in all, why would there be any benefits from a generalised model?
That is bad advice.
VGT Vanguard Technology ETF has outperformed S&P 500 over the past 20 years.
All the people who say “VTSAX and chill” disappeared in the past 3-4 years because their cherished total passive index fund is no longer the best over long horizons. And no, the markets are not efficient.
Given the techie audience here, I want to caution that investing in the same industry as your job is a kind of anti-diversification.
A really severe example would be all the people who worked at Enron and invested everything in Enron stock.
Even if your employer/investments aren't quite so fraudulent, You don't want to be in a situation where you are long-term unemployed and are forced "sell low" in order to meet immediate needs. If only one or the other is hit, you can ride things out more effectively.
- started to diff the executives statements from one quarter to another. Like engineering projects alot of this is pretty standard so the starting point is the last doc. Diffing allowed us to see what the executives added and thought was important and also showed what they removed. This worked well and for some things still does, this is what a warrant canary does, but stopped generating much alpha around 2010ish.
- simple sentiment. We started to count positive and negative words to build a poor mans sentiment analysis that could be done very quickly upon doc release to trade upon. worked great up until around 2013ish before it started to be gamed and even bankruptcy notices gave positive sentiment scores by this metric.
- sentiment models. Using proper models and not just positive and negative word counts we built sentiment models to read what the executives were saying. This worked well until about 2015/2016ish in my world view as by then executives carefully wrote out their remarks and had been coached to use only positive words. Worked until twitter killed the fire hose, and wasn't very reliable as reputable news accounts kept getting hacked. I remember i think AP new's account got hacked and reported a bombing at the white house that screwed up a few funds.
You also had Anne Hathaway news pushing up Berkshire Hathaway's share price type issues in this time period.
- there was a period here where we kept the same technology but used it everywhere from the twitter firehose to news articles to build a realtime sentiment model for companies and sectors. Not sure it generates much alpha due to garbage in, garbage out and data cleaning issues.
- LLMs, with about GPT2 we could build models to do the sentiment analysis for us, but they had to be built out of foundational models and trained inhouse due to context limitations. Again this has been gamed by executives so alot of the research that I know of is now targeted at ingesting the Financials of companies and being able to ask questions quickly without math and programming.
ie what are the top 5 firms in the consider discretionary space that are growing their earnings the fastest while not yet raising their dividends and whose share price hasn't kept up with their sectors average growth.
Looking back at that era it seemed investors were too focused on the numbers and fundamentals, even setting up live feeds of the factories to count the number of cars coming out and thats the same feeling I get from your post. It seems like dumb analysis ie. analysis without much context.
We now know from the recent Isaacson biography what was happening on the other side. The shorts failed to measure the clever unorthodox ways that Musk and co would take to get the delivery numbers up. For example: The famous Tent. Musk used a loophole in CA laws to set up a giant tent in the parking lot and allowed him to boost the production by eliminating entire bottlenecks from the factory design. There is also just the religious like fervor with which the employees wanted to beat the shorts. I dont think this can be measured no? It helped to get them past the finish line.
I've been on both sides of this trade, regularly.
Bear thesis back then was same as now. In retrospect, I give it a few more credits because Elon says they were getting close to bankrupt while he was posting "bankwupt" memes and selling short shorts.
Being a pessimist, and putting your money where your mouth is in markets, is difficult because you have to be right and have the right timing.
Oh boy... I wonder how a neural net trained with unsupervised learning has a predictive ability. I wonder where that comes from... Unfortunately, the article doesn't seem to reach a conclusion.
> We implement the CoT prompt as follows. We instruct the model to take on the role of a financial analyst whose task is to perform financial statement analysis. The model is then instructed to (i) identify notable changes in certain financial statement items, and (ii) compute key financial ratios without explicitly limiting the set of ratios that need to be computed. When calculating the ratios, we prompt the model to state the formulae first, and then perform simple computations. The model is also instructed to (iii) provide economic interpretations of the computed ratios.
Who will tell them how an LLM works and that the neural net does not calculate anything? It only predicts the next token in a sentence of a calculation if it's been loss-minimized for that specific calculation.
It looks like these authors are discovering large language models as if they are some alien animal. When they are mathematically describable and really not so mysterious prediction machines.
At least the article is fairly benign. It's about the type of article that would pass as research in my MBA school as well... It doesn't reach any groundbreaking conclusions except to demonstrate that the guys have "probed" the model. Which I think is good. It's uninformed but not very misleading.
You don't understand LLMs as well as you think you do. Yes, the neural network calculates things.
>It only predicts the next token in a sentence of a calculation if it's been loss-minimized for that specific calculation.
No that's not necessary at all.
https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mec...
https://cprimozic.net/blog/reverse-engineering-a-small-neura...
I do not think that SOTA LLMs demonstrate grokking for most math problems. While I am a bit surprised to read how little training is necessary to achieve grokking in a toy setting (one specific math problem), the domain of all math problems is much larger. Also, the complexity of an applied mathematics problem is much higher than a simple mod problem. That seems to be what the author of the first article you quoted thinks as well.
Our public models fail in that large domain a lot. For example, with tasks like counting elements in a set (words in a paragraph). Not to mention that they fail in complex applied mathematics tasks. If they have been loss-minimized for that specific calculation to the point that they exhibit this phase change, then that would be an exception.
But in the financial statement analysis article, the author says explicitly that there isn't a limitation on the types of math problems they ask the model to perform. This is very, very irregular, and there are no guarantees that model has generalized them. In fact, it is much more likely that it hasn't, in my opinion.
In any case, thank you again for the article. It's just such a massive contrast with the MBA article above.
The corrupt government officers then start using the AIs to try to cover up the evidence of their crimes in the financial statements. The AI possibly putting the skills of high-end and expensive human accountants (or better) into the hands of local governments.
Who wins this attrition war?
Bad actors will always exist but I think there's a LOT of genuine good to be done here!
The AI companies. Double. People paying to use their products. But mostly by gaining a lot of leverage and power.
You're making it way too complicated. The government will simply make AI illegal and claim it's for safety or something. The'll then use a bunch of scary words to demonize it, and their pals in the mainstream media will push it on low-information voters. California already has a bill in the works to do exactly this.
There's a difference between an AI being able to answer questions and it helping cover up evidence, unless you mean "using the AIs for advice on how to cover up evidence"
The ability to summarize and ask questions of arbitrarily complex texts is so far the best use case for LLMs -- and it's non-trivial. I'm ramping up a bunch of college intern devs and they're all using LLMs and the ramp up has been amazingly quick. The delta in ramp up speed between this and last summer is literally an order of magnitude difference and I think it is almost all LLM based.
That’s the root of your problem. Too many governments, not enough attention available to keep them accountable.
is there a demand for this. I live in cook country. I really don't want to ask these questions. Not sure what I get out of asking these questions other than anger and frustration.
And if not the rating agencies, people who invest in municipal bonds.
and of course, the follow-up questions. Like who.
Our major just appointed some pastor to a high level position in CTA( local train system) as some sort of patronage.
Thats the a level things operate in our govt here. I am skeptical that some sort of data enlightenment in citenzery via llm is what is need for change.
edit: looks like the pastor buckled today https://blockclubchicago.org/2024/05/24/pastor-criticized-fo...
This isn't meant to be overly negative, but exposing financial corruption is mostly about information control; I don't see how LLMs help much here. Even if/when you find slam-dunk evidence that corruption is occurring, it's generally very hard to provide evidence in a way that Joe Average can understand, and assuming you are a normal everyday citizen, it's extremely hard to get people to act.
As a prime example, this bit on the SF "alcohol rehab" program[0] went semi-viral earlier this week; there's no way to interpret $5 million/year spent on 55 clients as anything but "incompetence" at best and "grift and corruption" at worst. Yet there's no public outrage or people protesting on the streets of SF; it's already an afterthought in the minds of anyone who saw it. Is being able to query an LLM for this stuff going to make a difference?
[0] https://www.sfchronicle.com/politics/article/sf-free-alcohol...
That's still cheaper than sending them to prison!
> But San Francisco public health officials found that the city saved $1.7 million over six months from the managed alcohol program in reduced calls to emergency services, including emergency room visits and other hospital stays. In the six months after clients entered the managed alcohol program, public health officials said visits to the city’s sobering center dropped 92%, emergency room visits dropped more than 70%, and EMS calls and hospital visits were both cut in half.
> Previously, the city reported that just five residents who struggled with alcohol use disorder had cost more than $4 million in ambulance transports over a five-year period, with as many as 2,000 ambulance transports over that time. [emphasis mine]
> The San Francisco Fire Department said in a statement that the managed alcohol program has “has proven to be an incredibly impactful intervention” at reducing emergency service use for a “small but highly vulnerable population.”
Literally:
> It costs an average of about $106,000 per year to incarcerate an inmate in prison in California.
That is, if they vote in the first place - in that example I gave above of a corrupt mayor stealing millions (Tiffany Henyard of Dolton, IL), the voter turnout was only 15%.
Around half of adults in the US are financially illiterate.
paper https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3520684
thesis https://ora.ox.ac.uk/objects/uuid:a0aa6a5a-cfa4-40c0-a34c-08...
- https://huggingface.co/datasets/lamini/earnings-calls-qa
- https://huggingface.co/datasets/lamini/earnings-raw
- https://github.com/lamini-ai/lamini-earnings-calls/tree/main
I jest - a well contained and specified context is important for a paper.
But also, questions like "it would be interesting to also try this other thing" is how new papers get written.
But these models only seem to perform these jobs on the surface. Enough that companies will try them and waste resources. This will just hurt the bottom line optimizer shops and boost the professionals doing quality work on the long run.
LLMs could help a person learn to understand financial statements better, that is it.
There are not all these hidden gems in financial statements though that are being currently missed that language models are going to unearth.
Financial statements are already intentionally vague and often intentionally misleading.
Corporate CEO/PR/Marketing are already the masters of writing many words while saying absolutely nothing.
I am surprised at the results in the paper. The biggest red flag is that the researcher are not sure why there is predictive ability in LLMs. Maybe they didn't control for some lookahead bias.
Edit: assuming that they initially provide good predictions
The only area you absolutely can’t use LLMs is in sub ms latency
More substantively, LLMs are for linguistic tasks. That’s why I’m super super bullish (heh) on llms for decoding EEG data, and incredibly bearish on their ability to accurately model a corporation’s asset flow. I just don’t see how the confounding variables / motivating forces would be at all linguistic. This is basically using LLMs for super-advanced arithmetic
I wonder what the results would have been with still-anonymized but non-fully standardized statements.
Still though, impressive.
But up until that day, it will probably be cheaper.
Seems like an odd test for a large language model. There are tabular models out there
https://www.youtube.com/watch?v=VxxmzoZTRW4
there are other videos on their youtube channel on the more analytical aspect of it, I just decided to share the latest one.
It's been a hit and miss for now, depending on the model used (chatgpt/gemini/clause etc.) the results can vary somewhat.
If standardized LLM models are used to analyze statements, expect the statements to be massaged in ways that produce more favorable results from the LLM.
“78% of retail investor accounts lose money when trading CFDs with this provider.”
https://www.fca.org.uk/news/press-releases/fca-confirms-perm...
Of course, there are a lot more ways to lose money than CFDs.
Markets matter, and some speculation is useful, but the purpose of markets is not speculation. Obviously.
If you want to make some money get trained and get a good salary. Save your money in safe assets
if you want to get supper rich be super creative, ensure going broke will only effect you (i.e. do not do this while supporting a family), and found a firm. You will likely fail, but there is a chance of super wealth and a bigger chance of a wild ride that will be good for you
Trading from the perspective of greed runs the risk of total destruction. Putting you in jail, maybe. Bankruptcy if not too unlucky. Many people out of work because of your misallocation, and if you do not care about that I'm not interested in you
The financial system is a zero sum game. (The economy in general is not) There is always someone cleverer and they likely do not care if they crush you. International finance is a snake pit
Friends, look after friends. Maximise happiness. Be honest, be ethical, be safe
Live long and prosper
You mean like the good salary you get working for a trading firm?
I'm not sure what this comment you made is meant to be, but it reads like a blend of somebody who's high and a tik tok wellness influencer.
Trading is not a zero sum game in the sense you intend to suggest it is. It is 0 sum only if all participants have the same trading horizon.
The pool grows in the same way because it is linked to the economy. The markets are a variety of players with different requirements.
Most transactions occur between parties who have different horizons. Yes the hft makes money over 5s and the pension fund loses it. But the pension fund is looking at the return over the next year, so the small loss to the hft is just a cost of acquisition.
It is lots of fun. Very mathy. A nerd's dream.
Then the data. Oh the amount of data. 34 Gbit/s to get the full US options feed last I checked (someone posted that here I think). Much of the rest is kiddie stuff compared to dealing with that.
People can lament has much as they want that it drains the great minds: it is fun.
I didn't invent that game. Don't blame the players.
Arguably, the purpose of markets is part price discovery, part liquidity, and arguably mostly to support economic growth and stability by channeling funds from savers to those who can invest them productively.
> If you want to make some money get trained and get a good salary. Save your money in safe assets
Sure, as long as you're not into dynamism.
No, not all of finance is a zero-sum game. If you're connecting a buyer and seller that otherwise wouldn't have met, you provided value. Same for connecting them through time (in that you can e.g. help prevent somebody having to panic-sell their house from getting a suboptimal price).
Sure, there's speculation, nepotism, corruption; there are immoral and illegal market practices with no end, but you're making it sound like that's the entire purpose of finance, and not an undesirable byproduct.
Also, as if these only exist there, and not everywhere where there is power and money: Politics, business, even charity are not immune.
Starting a company is more ethical than trading – seriously? While there might be a general trend, can you think of no philanthropic traders and of no unethical founders (some of them in jail)?
> If you want to make some money get trained and get a good salary. Save your money in safe assets
100% agreement on the first part. But if everybody invests their money in "safe assets", there is no capital for people to start companies other than banks. Is that desirable? And who even determines what a safe asset is? What about people that manage and allocate risk? That's a function of finance again!
> Friends, look after friends. Maximise happiness. Be honest, be ethical, be safe
I agree, but this arguably has little to do with the remainder of your sweeping generalization.
Additionally, a wage is the same amount increase per month, while stock is in percentages.
Can't beat percentages depending on your wage/buy-in.
I don't think this is a signature of a person being honest, or trustworthy.