Compaction has been a pretty painful part of local llm usage. Scrapping the current context and parsing almosy 128k of context then generating something like 5-10k tokens - that can take quite a while when you’re working with 10t/s-45t/s (depending on the model).

I pretty much just start a new session whenever i fill the context.