I do like the trend.
Imagine if Anthropic or someone eventually release a Claude 3.5 but at like a whopping 10x its current speed.
Would be incredibly more useful and game changing than a slow o1 model that may or not be x percent smarter.
Imagine if Anthropic or someone eventually release a Claude 3.5 but at like a whopping 10x its current speed.
Would be incredibly more useful and game changing than a slow o1 model that may or not be x percent smarter.
https://inference.cerebras.ai/
It's pretty fast, but my understanding is that it is still too expensive even accounting for the speed-up.
And yeah their cost is ridiculous, on the order for high 6 to low 7 figures per wafer. The rack alone looks several times more expensive than the 8x NVIDIA pods [1]
[1] https://web.archive.org/web/20230812020202/https://www.youtu...
I think there's another one but I can't remember the name of it.
Also a bit further out is https://spectrum.ieee.org/superconducting-computer
"Instead of the transistor, the basic element in superconducting logic is the Josephson-junction."