You are ignoring a mountain of papers trying all conceivable approaches to create models. It is evolution by selection, in the end transformers won.
Can you help my little brain understand the problem by elaborating?
Also you may want to chill with the personal attacks.
"Maxima" is plural, "maximum" is singular. So you would say "a local maximum," or "several local maxima." Not "a local maxima" or, the one that really got me, "getting trapped in local maxima's."
As for the rest of it, carry on. Good discussion.
The plural is also used for indefinite number, so “the set of local maxima” remains correct even if the set has cardinality 1, but a member of the set has definite singular number irrespective of the cardinality of the set.
It is definitely NOT out of your reach to try any ideas you have. Kaggle and other sites exist to make it easy.
Good luck! 8)
Are we sure there’s anything “net new” to find within the same old x86 machines, within the same old axiomatic systems of the past?
Math is a few operations applied to carving up stuff and we believe we can do that infinitely in theory. So “all math that abides our axiomatic underpinnings” is valid regardless if we “prove it” or not.
Physical space we can exist in, a middle ground of reality we evolved just so to exist in, seems to be finite; I can’t just up and move to Titan or Mars. So our computers are coupled to the same constraints of observation and understanding as us.
What about daily life will be upended reconfirming decades old experiment? How is this not living in sunk cost fallacy?
When all you have is a hammer…
I’m reminded of Einstein’s quote about insanity.
Theoretical efficacy doesn’t guarantee real world efficacy.
I accept that this is self reinforcing but I favor real gains today over potentially larger gains in a potentially achievable future.
I also think we are learning practical lessons on the periphery of any application of AI that will apply if a mold-breaking solution becomes compelling.
They barely work for a lot of cases (i.e., anything where accuracy matters, despite the bubble's wishful thinking). It's likely that something will sunset them in the next few years.
People tend to favor single winners.
Those things aren’t even true in a 500 person company let alone an economy.
we're at the end?
Transformers have first mover advantage. They were the first models that scaled to large parameter counts.
That doesn’t mean they’re the best or that they’ve won, just that they were the first to get big (literally and metaphorically)
The fact that one person (initially) was able to do it highlights how much low hanging fruit there is for non transformers.
Also, the fact that a small number of people designed, trained, and published 5 versions of a perfectly serviceable (as in has decent summarizing ability. The biggest LLM use case) model which doesn’t have the time complexity of transformers is a big deal.