Of course this dog and pony show just got NVIDIA past $3 trillion, it's like crypto on steroids.
Just for reference, this is from nine years ago:
And this is from five years ago:
"You may not need hospitalization for respiratory problems for which there is pain or even full internal combustion. Breathing is particularly hard in the high-pitched high pitch, low pitched pitches. Air you inhale and exhale can cause a shortening in your breathing, so a ventilator is best. Doping can be administered on your own. If you fall, you must be at least 1 kilometer away from you to be considered for the testing. Your turn-of-the-seat's softerness may be a sign that something has gone wrong. Your depth should not exceed the mean of your seat height when wearing the seatbelt."
And for two... LLMs are also getting to be pretty old. BERT and GPT-2 are both more than 5 years old, and still have fundamental issues that we can't be sure are solvable. They lie confidently, they conflate facts and fiction, they create entirely made-up scenarios when asked about real-world happenings. It should be extremely alarming that the only successful AI strategy to-date has been scaling-up an already inefficient model.
AI will improve with time, but it seems entirely plausible to me that we've already hit the proverbial "bathtub curve" of progress. I genuinely cannot imagine what a "generational leap" in LLM technology would look like, outside of fixing the hallucination issues.
You sound like you have no clue that five years are the blink of an eye.
Cryptocurrency apologists also used this line of logic. "Yes, today the value of crypto is nothing... but imagine where it will be in five years!" Then we all wait 5 years, and cryptocurrency is still the victim of it's own mindset. Like cryptocurrency, I think LLMs have "technically" solved what they set out to do; generate readable text. But you need more than a solution in search of a problem; there isn't really that much demand for marginally truthful text in the same way completely decentralized currency isn't really necessary for the average well-meaning civilian. I'd even go further and argue that introducing AI into our daily lives requires you to replace something else, something that was probably more accurate and better-designed than the AI replacing it.
If 5 years is a "blink of an eye" for the industry, the entire field will be dead within 18 months. VC just moves that fast.
You clearly have some very specific models in mind. Even if the latest 4B and 8B models don’t move the needle on the “results you would champion” metric, this does not advance your argument that the state of the art hasn’t significantly progressed from 5 years ago.
> I would legitimately argue
I’ll bet you would!
Because this to you is "generate readable text"?
https://m.youtube.com/watch?v=MirzFk_DSiI
Sorry to be so direct, but you're in denial.
There's a reason why GPT-4o is not taking over YouTube and social media with it's incredible capabilities; nobody cares.
Well, if it encodes a world model, and can work on its own encoding, loop it into a critical revision of its contents and you have the Real Thing - something that is informed and reasons.
That a full world model is there is the preliminary condition. So, first you have to establish that an LLM actually draws a world model in its inner structure.
Then you have to refine that world model, and be able to refine its intentionally spawned branches (e.g. "Imagine scenario X"...). If the iteration of the refining is successful, you achieved the goal.
The point is implementing critical thinking; that the initial world model is flawed is the known starting point: at the beginning, it only knows it has "heard" a lot of information.
If it has no concrete reasoning, how is this looped revision intended to be more-accurate than a zero-shot guess?
I agree with the fundamental principle of not trusting input data from the start, but that just puts us back in square-one where we generate non-authoritative results dependent on a series of autoregressive parameters. It might generate better-reasoned results, but the outcome will be equally unreliable and prone to hallucination.
It can work if the reasoning (the logic) is part of the world model and is similarly refined.
Then, like in the normal case, you gather a good amount of information than draw your conclusions based on you logic and the "wisdom" built. Were this automated, you would have an automated champion in judgement - doing its best with the imperfect data and function available. Like us, just better.
We started a very long time ago and you have to give it time. The current turn of events seems to have turned its back on some very critical aspects - and still, give it time.
Impatience at this stage seems very ill-posed.
Although you probably meant that "LLMs show no signs to "emerge" (through upscaling) what they should become".