Increase GPT4's context size by asking it to compress your prompts

If you look into this into more detail:

A) people have had a hard time reproducing it, and

B) more damning, the "compressed" version uses more tokens than the original (https://gist.github.com/VictorTaelin/d293328f75291b23e203e9d...)

> In practice, plain, well-designed summaries should be optimal to fit larger documents in the context.

> This concept has potential, though; building lookup tables seems to outperform long text summarization.

It's a clever idea, and I agree that lookup tables and external storage of memory is likely going to be important at some point, but I suspect that's going to come out of giving LLM more ability to externally reference "long-term" memory rather than compressing everything into immediate context.

A more accurate title would be "decrease GPT4's context size by asking it to obfuscate your prompts". You're getting a prompt that consumes more tokens and isn't particularly faithful to the original.

One reply points out that even though the text length is 1/4 of the original, the “compressed” output is actually LONGER in token count!

https://twitter.com/dogeofcoin/status/1642918892602290179

A promising and potentially useful approach for adding additional context to your prompts.

How likely is it that context size will greatly increase in the coming year or two? Are there fundamental limits, or could we reasonably expect greatly increased context size in the future?

ChatGPT 3.5 in response to a brief analysis of the community gave out: "It seems like the conversation you provided is discussing the idea of compressing prompts or summaries of larger documents in order to fit them into a smaller context. While the idea has potential, it appears that people have had difficulty reproducing it and that lookup tables and external storage of memory may be more effective in the long term. Additionally, there is discussion about the possibility of greatly increasing context size in the future, with the example given that there was an almost 10x jump between GPT-3.5-turbo and GPT-4 in terms of token capacity."

The compressed one is shorter number of characters but does it have fewer tokens? https://platform.openai.com/tokenizer

The context length includes all of GPT4's replies, is that correct? If I want to minimize the context length, should I be asking GPT4 to limit its replies to just the fix that is requested?

GPT4 seems to prefer long-winded replies, i.e., when I ask for what amounts to a one-line fix, it repeats the entire block of code with the one line correction. In contrast, Replit's Ghostwriter often gives a concise reply showing only the one-line fix, and I have to ask for the entire block when it isn't clear where the fix is applied.

This post from @gfodor is also a really interesting read on the topic of compressing prompts, not sure what the main use case would be though!

https://twitter.com/gfodor/status/1643444605332099072?s=20

ZipGPT is a novel, LLM-based non-deterministic compression algorithm. Give our beta version a try. The results may surprise you!

This is the most disappointing part about the frenzy.

Obvious ideas being held up as something more than that.

If you think this is incredible, give yourself some time to go play with LLMs. Try things. Clever things. Stupid things.

It helps set a more reasonable scale for these sensational sounding snippets.