Remarkably, foundation models can learn new tasks via a few examples (called few-shot learning). LLM answers also significantly improve when given relevant supplemental information. Boosting context length: grows its "working memory"; provides richer knowledge to inform its reasoning; and expands its capacity for new tasks, given germane examples.
> Would virtually unlimited context move us much closer to an LLM where training is continuous?
No. You can already continually train a context-limited LLM. Virtually unlimited context window schemes also exist. Training is separate concept from context length. In pre-training, we work backward from a model's incorrect answer, tweaking its parameters to more likely say the correct thing next time. Fine-tuning is the same, but focusing on specific tasks important to the user. After training, when running the model (called inference), you can change the context length to suit your needs and tradeoffs.