Hacker News Reader

Comment by aperrien | Hacker News Reader

To me, it much more closely resembles a key-value retrieval system. I wonder if we could learn anything about that by training an artificial neural net to perform key-value storage?

View on HN

Machine Learning phd student here! I am currently working on a branch of ML called Continual Learning that deals with this theme exactly. It turns out Artificial Neural Networks have a hard time recalling things they they see in the past [1].

One of the most effective strategies to counter this issue is storing and periodically replaying old training items (i.e. “memories”) for the network [2]. There are also approaches that set out to mimic what we understand of the brain and adopt cue-based approaches [3], but they don’t seem to have reached meaningful outbreaks just yet...

[1] https://en.wikipedia.org/wiki/Catastrophic_interference [2] https://arxiv.org/abs/1902.10486 [3] https://arxiv.org/abs/1711.10563

I think it's more like a graph with edges between associated bits of information. It's easy to remember things in order, but random access is hard unless you've established links for it. For example, if you memorized how to play a song, each part links to the next but harder to ask someone what is the n-th note in a song, even if they can play the song. It would explain why making associations is a useful memory technique. Traversing this graph when you aren't near what you're looking for can be hard ("it's on the tip of my tongue").

Not only that, but sometimes as happened to me recently with my own song, you completely forget how to play it until you hear it and start playing it, then suddenly you remember every little note and nuance you do did, but only while you're playing it and good luck explaining to someone what you're doing even as you're doing it.

It's odd, because it very much resembles relational memory for me. I can summon no memory just by it's name, only from it's association. The space for sole base values is very limited. The plus side is that i can generally remember every caveat and detail there is to doing a thing once I've seen it once.

Yes, this idea is called sparse distributed memory https://en.m.wikipedia.org/wiki/Sparse_distributed_memory

Yeah, my brain can prolly be 80% modeled as a Redis fork...

Content addressable memory more like.

I’ve been saying for years that it’s like a hashtable, where the key itself is hashed in a predictable quickly retrievable pattern or namespace

This article does not invalidate the idea

Reply on news.ycombinator.com