Was going to point out the same. Glad to have the paper to read but I don't think the findings are significant.
That's not to suggest that Renaissance is going to start using Chat GPT tomorrow, but maybe in a few years they'll be using fine tuned versions of LLMs in addition to whatever they're doing today.
Even if it's not going to compete with the state of the art models for something, a single model capable of many things is still useful, and demonstrating domains where they are applicable (if not state of the art) is still beneficial.
"In a few years" you'd have the benefit of the current, bespoke tools, plus all the work you've put into improving them in the meantime.
And the LLM would still be behind, unless you believe that at some point in the future, a radically better solution will simply emerge from the model.
That is, the bet is that at some point, magic emerges from the machine that renders all domain-specialist tooling irrelevant, and one or two general AI companies can hoover up all sorts of areas of specialism. And in the meantime, they get all the investment money.
Why is it that we wouldn't trust a generalist over a specialist in any walk of life, but in AI we expect one day to be able to?
The specialist is a result of his general intelligence though.
I have a slightly more cynical take: Those LLMs are not actually general models, but niche specialists on correlated text-fragments.
This means human exuberance is riding on the (questionable) idea that a really good text-correlation specialist can effectively impersonate a general AI.
Even worse: Some people assume an exceptional text-specialist model will effectively meta-impersonate a generalist model impersonating a different kind of specialist!
Eloquently put :-)
If there were some super generalist that could then the specialist would have no power.
"I didn't violate a red light. I wasn't even driving, the AI was!"
"The AI said you did, that's 50,000 yuan please."
So it all needs checking. It's the classic LLM situation. If you're trained enough to spot the errors, the analysis wouldn't take you much time in the first place. And if you're not trained enough to spot the errors...
And let's say it does work. It's like automated exchange betting robots. As soon as everyone has access to a robot that can exploit some hidden pattern in the data for a tiny marginal gain, the price changes and the gain collapses.
So if everyone has the same access to the same banal, general analysis tools, you know what's going to happen: the advantage disappears.
All in all, why would there be any benefits from a generalised model?
That is bad advice.
VGT Vanguard Technology ETF has outperformed S&P 500 over the past 20 years.
All the people who say “VTSAX and chill” disappeared in the past 3-4 years because their cherished total passive index fund is no longer the best over long horizons. And no, the markets are not efficient.
Given the techie audience here, I want to caution that investing in the same industry as your job is a kind of anti-diversification.
A really severe example would be all the people who worked at Enron and invested everything in Enron stock.
Even if your employer/investments aren't quite so fraudulent, You don't want to be in a situation where you are long-term unemployed and are forced "sell low" in order to meet immediate needs. If only one or the other is hit, you can ride things out more effectively.