Hacker News Reader

Comment by nojs | Hacker News Reader

> Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

Based on a human output rate of 3.3 tok/s, which seems questionable as a means of comparison

I do believe that this is the trade off. We are more efficient but slower in terms of thinking (at the same level of intelligence). Some animals go much further in terms of that trade off, see https://en.wikipedia.org/wiki/Portia_(spider) for example.

Just checked wikipedia page... they have like 100K neurons only.. wtf!!! How can nature cramp all senses, including spatial, motion, life maintenance and general thinking into just 100K neurons?

A lot of the low level stuff is outsourced to biochemistry: the physical properties of proteins, and the various self-regulating biochemical systems of an animal, can "encode" a lot of intelligence, easing up on the computational demands of the brain proper.

Hmm... is there something that can estimate how many bits of information in portia, each neuron+synapse combination might be encoding?

100k is already a lot. Plenty of insects get by with a couple of thousand, and pack a whole bunch of complex behaviour, including flight, into that.

Our brain is more like a MoE model activating only a few neurons for specific activities making it more efficient unlike a dense model activating all the params.

Also brain produces quality tokens @ 3.3 tps instead of fast generating hallucinated tokens by certain models. Thus MTP can produce low quality tokens at 2x speed.

Patience pays.

What exactly are you questioning?

The claim that tok/s independent of quality is a useful comparison (I can get thousands of tok/s on a suitable small model), and secondarily that humans can’t output “tokens” faster than than in some sense, which I am less confident about

Reply on news.ycombinator.com