> Why does it matter if that output is stored in the LLM's context

Context window is expensive and precious. Much better to offload to some medium where it isn’t.