Hacker News Reader

An HTM layer is one feedforward layer in a Heirarchical Temporal Memory network. See for example https://arxiv.org/abs/1511.00083 which trains one-layer HTM models. See for example https://timallanwheeler.com/blog/2022/01/09/hierarchical-tem... for a more approachable blog post about the same topic

View on HN

That's a very nice blog post, thanks!

I'm not sure why you think HTM layers are bigger than modern DL layers. The HTM layer configuration used in the paper (B=128, M=32, N=2048, and K=40) is 335M parameters. Compare to GPT-3 with 96 layers, where each layer has 1.8B parameters. Much larger models than GPT-3 have already appeared with no end in sight as to how much more they can scale.

The point is, if HTM worked, people would throw compute resources at it, just like they do with DL models. But it doesn't.

Reply on news.ycombinator.com