86 karma · joined September 14, 2024
Also the state of SIMD in Cranelift is also very WIP. They pretty much just support a subset of 128bit vectors with some rare exceptions.
While qwen3-4b is smarter than I remembered it, newer models seem to be way better, K2 Horizon 3.7B for example. Is there a reason why that qwen model was choosen?
My last question would be if the same optimizations could be made to a Jev-like model. I mean Jev is already fast and probably has a high throughput per H100. So could we maybe get to >2B tokens/minute/GPU with such a model?
Didn't know that WG can't use hardware crypto accelerators. I hope it will someday
But it can apparently also run 5.6 Sol
The "original" X Elite was also supposed to get good linux support but it unfortunately never happened.
So hopefully the X2 will actually happen!
I kind of hated Astra for it's poor instruction following and stopping all the time plus bad code quality. It somehow feels a bit like some of the popular open models but with a lot more knowledge or peek capability. But it doesn't reach peek that often