As I understand it, they would have to train a whole knew model to hard cap it's context to different lengths. That would be cheaper to train and had cheaper inf, but still a huge investment.
So I'd guess it's API level.
So I'd guess it's API level.
Kimi K3 supports a context window of up to 1 million tokens. We achieve this through extending
the context window progressively as training proceeds, following a four-stage curriculum. The
window grows from 8K to 64K tokens during pre-training, and from 256K to 1M tokens during the
cooldown phase.
With a lot of architectures, you technically don't need to retrain to extend the context window either; e.g. RoPE scaling; but performance is typically crap.