Hacker News Reader

Comment by benbreen | Hacker News Reader

Forgetting context is a problem, for sure. One thing I've found that works fairly well is to include a request for a "status bar" in your prompt. I.e. you ask it to remind itself with each response 1) who it is pretending to be 2) what the date is 3) what the setting is 4) what is in their NPCs "inventory" (which it intuitively understands because LLMs seem to have a natural affinity with MUDs). You can even have it track its mood and variables like weather.

As the context windows of Claude/GPT-4 etc increase, I think this will be less of an issue, but for now it's a pretty effective workaround.

Here's an example of the prompts I'm using (from an activity I just did with my world history class): https://docs.google.com/document/d/1sLRsUVJ_KSPtjrO83ko2MSFf...

And my writeup of an earlier version: https://resobscura.substack.com/p/simulating-history-with-ch...

View on HN

I played through the "fall of the ming dynasty" one when I saw your submission and it was pretty fun. I do find they are a little too easy to persuade of heroic things. It's easy to lead a heroic resistance and win the fight within a turn or two if you provide a justification, no matter how tenuous or unlikely. Especially if it's "feelgood" like you stand on a table and make a speech about how the wealthy kleptocrat emperor should agree to give the peasants their freedom if they resist the invaders or whatever (even if that makes no sense and would never work)

I'm not sure how exactly ChatGPT uses its own responses in the context, but I've found that it can ignore the instructions you've made it repeat over and over, even though it just did it successfully in the last response. I've had better results making it produce something to paste in my next prompt.

Relatedly, when editing past prompts, I've found that it sometimes reuses things that were in the original branch of the conversation, even though they're not in the logical flow of the current conversation.

NovelAI uses something like this with their home-rolled models: you can put in some generic 'memory' as well as keyword-based notes and it will automatically get included up to a certain token limit, including some prioritization handling for cases where many different notes are triggered at once.

A useful 1a for ChatGPT specifically is BabyAGI-style "what their current/next objective is."

There are vector database solutions that address the forgetting of context. Oobabooga has superbooga (chromadb), SillyTavern requires the headache of the Extras service but will let you use chromadb, Google or OpenAI. Koboldcpp doesn't do vectorization, but does have a clunky autogenerate-summary feature that injects a summary of events so far into the prompt.

Reply on news.ycombinator.com