Hacker News Reader

TTT, cannon layers, and titans seem like a stronger approach IMO.

Information needs to be compressed into latent space or it becomes computationally intractable

do you have references to

> TTT, cannon layers, and titans

This was the nearest reference I could find. Links to an unofficial pytorch implementation on Github are also linked in the threads somewhere - https://www.reddit.com/r/LocalLLaMA/comments/1i0q8nw/titans_...

is titans replicated? I feel like lucidrains couldn't replicate.

I think something like Titans explains Gemini's excellent long context performance. That would explain why the Titan team hasn't released the training code or hyperpameters used even though they said in the paper that they would, and why soon after that it came out that DeepMind would be holding off publishing new results for 6 months to avoid giving away competitive advantages.

Reply on news.ycombinator.com