Was expecting the article to go more in-depth.
Say, what happens when chain of summaries grows so long, that it still overflows context window. Is summarization runned over the summaries in the context window?
Say, what happens when chain of summaries grows so long, that it still overflows context window. Is summarization runned over the summaries in the context window?
1. What happens if it overflows during assistant's turn, while it makes tool calls? Is it better to make a compaction in the middle of the chain of tool calls, or maybe before a potentially long chain of tool calls? If latter, how to choose the right point for compaction?
2. There should be enough space in the context window for a summary. What if, theoretically, a single new user message already overflows the context window? Or what happens if a summary is too long?
Maybe those are stupid questions, but I'm making a point that there is a room for going in-depth.