Hacker News Reader

The largest model they tested is 1.7B.

Do they believe that it's likely to scale as high as "standard" LLMs? Or is this another state space machine moment where it turns out to be equivalent to them?

The concept of a memory module, separate from a prediction engine, makes sense to me, our brains might operate like that (short vs long term memory).

Many research groups have been trying to implement this idea, recent paper from Meta comes to mind: https://arxiv.org/html/2412.09764

If they believe it’s likely to scale, they will try to scale it, so we will either hear about it in a few months, or we won’t.

Reply on news.ycombinator.com