Hacker News Reader

Comment by Qu3tzal | Hacker News Reader

5-8 layers but how many weights per layer? I can’t access the article but it says it’s a deep CNN so probably in the order of a few thousands weights per layer.

View on HN

Up to 256.

Is that 256 weights per-cell per-layer, or 256 in total? Is "how many cells in each layer" a valid question too?

Their pretrained 128-wide net has a total of 9.2 million parameters: https://www.kaggle.com/selfishgene/single-neuron-as-deep-net...

9.2 million parameters per neuron. There an estimated 86 billion neurons in the human brain [0] and 19 billion in the neocortex [1]. That means that, for this strategy to emulate a human brain or neocortex would require 791 quadrillion parameters or 175 quadrillion parameters, respectively. The largest ANN built so far, GPT-3, has 175 billion parameters [2]. We are 6 orders of magnitude from being able to pull it off.

[0]: https://en.wikipedia.org/wiki/Human_brain

[1]: https://pubmed.ncbi.nlm.nih.gov/9215725/

[2]: https://siliconangle.com/2021/08/04/microsoft-researchers-re...

This is assuming there's no weight sharing.

Some of those weights presumably go to reproducing highly-conserved features, like the kinetics of particular ion channels. These are "tied" via the genome, in the sense that there's one KCNC1 gene, but millions of neurons express the ion channel it encodes.

On the other hand, this model is also missing all sorts of other interactions: hormones and other neuromodulators, ephaptic coupling, etc.

It's so complicated I would venture that no one even has a reasonable guesstimate of how close we are, beyond "Not very."

Reply on news.ycombinator.com