I'm somewhat doubtful that we will be seeing something as large as Kimi K3 in silicon any time soon.

This tech can definitely scale up from the current 8B prototype, but - at least as far as my limited understanding of the tech involved goes - you cannot just ASIC a trillion weights model due to physical size constraints.

___

Specification HC1

Model Llama 3.1 8B (hardwired)

Process TSMC 6nm

Die size 815mm²

___

So the current prototype already pushes the limits of what we can fit on a single die, and that is already likely going to limit your yield.