Or perhaps a progressive summary, where the most recent messages are full fidelity, and older messages get “compressed” into a summary.

You can also fine tune the model to incorporate larger amounts of data, but that may be more expensive (and slower)

This kind of sounds like human short term and long term memory. Maybe “fine tuning” is analogous what happens to our memory when we sleep.