Mental time travel: Hierarchical memory for reinforcement learning agents (2021)

Fun coincidence, I read this paper as part of my AI research just recently. It's actually quite neat! Mostly because there is really not much like it in the sense that it focuses on exploring a memory mechanism for RL agents - most of the time people just stack frames or slap on an RNN and call it a day. The set of tasks evaluated on is cool too.

> . HCAM stores memories by dividing the past into chunks, and recalls by first performing high-level attention over coarse summaries of the chunks, and then performing detailed attention within only the most relevant chunks.

This sounds just like the techniques the human 'memory athletics' gurus teach.

> or reasoning over memory

Where is the "reasoning" in this? sounds like pattern recognition to me.

The authors are seemingly falling into the fallacy of replicating the human experience, which simply leads to ultimately inefficient Alt Intelligence. While an interesting idea which could have a couple very rare useful use-cases, this seems like another of the million [Deep Learning + a minor wacky idea] research papers.