That is to say, specifically, that it is not possible, by any means, to take any block of text and reduce it to a smaller block of text without losing some information.
That is infinite compression; if you could do this, you could reduce any dataset to 1 bit and recover the data seamlessly.
You can’t.
What that means is that this approach is fundamentally just improving performance, it’s not solving the problem.
When you compress a conversation into a summary, some information will be lost. You can tweak and fold and do whatever clever things you want, but fundamentally you are losing information.
…and the process is recursive. So at some point you will be summarising a set of summaries… and you will lose some information you’ve summarised, to some extent.
You can’t not lose information with this process.
So… while, it probably helps in trivial cases, putting recursive summaries into your prompt is kind of daft, and almost certainly doesn’t actually work when you try use it to do useful things.
It probably looks like it’s working when you don’t use the recursive summaries heavily, because at that point you’re not losing much information.
…but, I’m going to bet that it doesn’t scale, and people using it will find that out pretty quickly as they use it.
This sounds trite/heretic but it is as fundamental in its nature as the concept of "incompressible data". I agree that you cannot spare any data in the absolute, but I argue that you can communicate information with less bits than the data holds if the peers and the "model" incorporates knowledge about what kind of data is being transmitted.
One bit of information can mean more the more parameters an LLM has. The text "a8ckd" could easily mean the entire Declaration of Independence or whatever to an LLM (it doesn't, of course).
It has memorised a lot of stuff and similar to us "bubble sort" means
// An optimized version of Bubble Sort
void bubbleSort(int arr[], int n)
{
int i, j;
bool swapped;
for (i = 0; i < n - 1; i++) {
swapped = false;
for (j = 0; j < n - i - 1; j++) {
if (arr[j] > arr[j + 1]) {
swap(&arr[j], &arr[j + 1]);
swapped = true;
}
}
// If no two elements were swapped by inner loop,
// then break
if (swapped == false)
break;
}
}
The more information stored in its parameters, the more information a single bit can hold.
All we need to do is find out how many times can you recur until it breaks down? 3 times? 5 times? Just a few times is still genuinely very useful!
At each summarization step you can go through and find which Q/A pairs have the highest relevance across all of the other questions in the current batch, which helps solve LLMs tendencies to get stuck in high similar repetitions