Who is trying to make time series forecasting with LLMs/tensors? I have seen like four posts in the last few days. I know LLMs/transformers are really cool but I do not think they are the silver bullet that solves this.
It's "LLMs can do this category of thing, too". The generality of the architecture is a novel scientific finding in and of itself. Do you remember the last 50 years of AI research where GOFAI systems were universally too fragile to function outside of highly constrained environments, and earlier neural networks were either ineffectual or extremely specialised? The fact that LLMs can function incredibly well (compared to historical AI systems) in diverse scenarios is the scientifically interesting finding.
Classical AI was about some genius trying to devise the most genius computer inference algorithm. But any single genius can only do so much in limited time and it is difficult to make them work together.
Whereas LLMs are not genius, they are simply using statistics to predict what is the accumulated intellect on the internet. They are simple but they combine the output of millions of minds, which no human genius could ever do.
They are a brain addition machine, rather than super-intelligent by themselves. But when you add together the outputs of many minds you get powerful general purpose output.
I like that metaphor. LLMs are masters of linguistic addition. They are literally made by adding up language gradients. They accumulate knowledge and insight through exposure to vast swaths of text, stitching together concepts and ideas into an ever-growing tapestry of understanding.
But whereas brains are isolated islands, unable to directly share their contents, language forms a collective reservoir that flows between minds. Words and ideas mix and mingle within this pool, combining into new formulations that reflect the present.
In this way, language displays an evolutionary dynamism that outpaces biological change. LLMs ride this wave, leveraging the emergent intelligence inherent in humanity's shared linguistic legacy. The wisdom accumulated over generations surpasses the capacity of any single mind. We stand upon the shoulders of giants, supported by the communal scaffolding of language and knowledge that previous generations erected. LLMs tap into this source, channeling and distilling the experience contained within our words.
tl;dr AI is riding the language exponential while we are distracted by models and brains, implementation details.
Accurate information is also over represented because you can be wrong in far more ways than you can be correct. In other words the wisdom of the crowd is in part the 10% of people that know X all match each other while the other 90% is all over the place.
I also don’t think you need to remove the dumb comments from the training material anyway. The AI could be able to simulate either someone of 80 IQ or 180 IQ, producing output consistent with both personas.
And ignorant person could have a very high IQ still be very ignorant.
LLMs may not be "smart" but they know a lot.
Yet, there exist problems where even old events retain their strength. An example is where we want to take into account discrete events (tokens in LLM) for predicting stock prices. These events might be explicitly defined (holidays, company announcements, important economic figures etc.) or derived from the data like technical patterns. The strength of transformers are in their ability to ignore the order of events and ignore the distances between them. More precisely, transformers can learn when it is important. In language models, this is used to generate output sequences where semantically equal tokens have completely different order than in the input sequence. Something similar can be done in time series forecasting if we accordingly define "tokens", for example, as technical patterns. Then rising stock prices can be explained (and predicted) not only because of recent numeric behavior but also because "something happened" two weeks ago.
[0] https://github.com/asavinov/intelligent-trading-bot Intelligent Trading Bot: Automatically generating signals and trading based on machine learning and feature engineering