After the trough of disillusionment comes the plateau of productivity.
All those previous silver bullets are still with us. This one's not going away either.
All those previous silver bullets are still with us. This one's not going away either.
When Taalas burned Llama 3.1 to the HC1 they saw at least an order of magnitude more tokens per joule. That saving is sitting right there, today, ready for anyone who is willing to say "this is the model we're freezing on for a couple of years".
It also seems plausible that it will take many years until hardware price/performance reaches a point where companies can provide models people deem acceptable at unsubsidized prices people deem acceptable. I don't know.
Take a look at this: https://en.wikipedia.org/wiki/Koomey%27s_law
2.6 years is about 16x in a decade. That's another path out - inference on GPUs gets so cheap they become profitable at current prices.
This is a lot more like the PC or the Internet than cryptocurrency.
Huh.