Given the current best open-weight model (Kimi 2.6) is 1T A32B, I wonder how long we’ll have to wait for hardware like strix halo or gdx spark to be able to run it.
In that sense, how long you'd need to wait to get say ~20tk/s .. maybe never.
(save a significant firmware update / translation layer)
Speculative decoding/DFlash will help with it, but YMMV.
Edit: Missed a part that this is A32B MoE, which means it drastically reduces amount of reads needed. Seems 20 t/s should be doable with 1TB/s memory (like 3090)