4,870 karma · joined September 7, 2021
At least now MiMo v2.5 exists and can be used as another dirt cheap multimodal model.
Does Boko Haram and ISWAP even control a single town or they just control a few villages in Lake Chad and in the Sambisa forest?
Also reading the report they seem quite clueless.
So the diffusion process takes more GFLOPs, if you have enough users you can already balance memory and compute.
What's novel here is the extremely small KV cache memory usage per long context windows, like 0.77GB with 512K, a 90% memory usage reduction compare to the already really small KV cache memory usage of Deepseek V4 Flash.
Don't want to ruin it but go read some old posts from the author about AI, the tone is the same and he is very much wrong.
Isn't that just projecting the patches into the d_model size vectors that the models takes?
>I am assuming that involves of quantization
12B model in 16GB seems very reasonable to me, int8 is top quality for running models.
Where does it feel vibed?
Edit: I noticed later it was in Milan, I guess it makes perfect sense.
AGI should at least match, not surpass humans in every cognitive task.