Hacker News Reader

Examples of such architectures:

RETRO (Retrieval-Enhanced Transformer) https://arxiv.org/abs/2112.04426

And much older (2016): Differential Neural Network https://www.deepmind.com/blog/differentiable-neural-computer...

RETRO has external source of data but is something entirely different to what is discussed in the paper. In simplistic terms, in RETRO architecture you store embeddings in an external database and then use kNN search. It is not an end-to-end model capable of things like those with stack or tape memory (what OP asked about), retrieval models are only good for limited Q/A atm.

Reply on news.ycombinator.com