Maybe it doesn't really need to fully remember it in order to still do a pretty ok job? I think its ~50 pages of context?