The unbearable slowness of being: Why do we live at 10 bits/s?

It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer.

When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative to each other. You also pick up the fact that it's a Rubik's cube and not a bird or a series of binary digits or English text. If an orange cat lunged at your Rubik's cube while you were studying it, you wouldn't process it as "face 3 has 4 red squares on the first row, then an orange diagonal with sharp claws", you'd process it as "fast moving sharp clawed orange cat attacking cube". Which implies that every time you loom at the cube you also notice that it's still a cube and not any of the millions of other objects you can recognize, adding many more bits of information.

Similarly, when you're typing English text, you're not just encoding information from your brain into English text, you're also deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk. Not to mention the precise mechanical control of your muscles to achieve the requisite movements, which we're having significant trouble programming into a robot.

> If the questions are properly designed, each will reveal 1 bit of information about the mystery thing. If the guesser wins routinely, this suggests that the thinker can access about 2^20 ≈ 1 million possible items in the few seconds allotted. So the speed of thinking – with no constraints imposed – corresponds to 20 bits of information over a few seconds: a rate of 10 bits per second or less.

This is an extrinsic definition of "information" which is task relative, and has little to do with any intrinsic processing rate (if such a thing can even be defined for the imagination).

The question of why does biological hardware capable of very high "intrinsic rates" deliver problem solving at "very low extrinsic rates" seems quite trivial. Its even a non-sequitur to compare them: properties of the parts are not properties of wholes. "Why does a gas move at 1 m/s, when its molecules move at 1000s m/s..."

All the 'intrinsic processing' of intelligence is concerned with deploying a very large array of cognitive skills (imagination, coordination, planning, etc.) that are fully general. Any given task has requires all of those top be in operation, and so we expect a much slower rate of 'extrinsic information processing'.

Consider how foolish the paper is to compare the intrinsic processing of a wifi network with the extrinsic task-specific processing of a human: it is likewise the case that if we set a computer the challenge of coordinating the solution of a task (eg., involving several LLMs) across a network, it's task-specific performance would drop off a cliff -- having a much slower 'solution rate' than 10bit/second.

These 'task-specific bits' represent a vast amount of processing work to solve a problem. And are at least as much to do with the problem, than the system solving it.

It seems to me all this paper does is define tasks in a highly abstract way that imposes a uniform cost to process '1 bit of task information'. Do the same for computers, and you'd likewise find tiny bitrates. The rate at which a problem is solved is 'one part of that problem per second' for a suitable definiton of 'part'

> Why can we only think about one thing at a time?

Maybe this is just a perception thing. Sure, you can only really keep up one stream of thought, visualization or inner dialogue (whatever you want to call it) at a time, but perhaps that's because we learn all our lives that direct communication is a one-channel, linear thing--speaking and listening focused on one topic at a time. Our brain does plenty of thinking in the background that leads to "a-ha!" moments even when the direct focus of our thoughts isn't on that topic. What if the mind could maintain multiple threads of thoughts at once, but our language coerces our thought patterns into being linear and non-concurrent?

Where do they get 10 bits/second?

Heck, I can type way faster than 10 bits per second, even after gzipping the output.

And when I consider the amount of sensory information that I consciously process (not that comes in, but that I conceptually analyze), it's got to be way higher.

10 bits/s doesn't pass the smell test.

I beg you, please read the paper before commenting. It's very interesting and it answers a lot of questions that might arise from just skimming the title.

At what speed is the brain processing when a pair of Chinese ping pong masters are playing? All of this requires seeing the ball coming from the opponent, subconsciously calculating its speed, angle, and rotation to decide how to move your body, legs, arms, hands, and even fingers to hit the ball at the desired location, with the desired angle, and at the desired speed. A standard ping pong table is 2.74 meters long, and in high-speed professional games, the ball can traverse this distance in as little as 90–140 milliseconds. This means players have less than a tenth of a second to react. It's not just the immense image-processing speed required but also the mental speed to transform that visual data into precise positioning and movement of hundreds of muscles in less than a second. Ten bits per second? No way.

The only time 'bits' will ever be an appropriate measure of human processing is when we are processing or producing diginal information artifacts, e.g. a typewritten document.

Our bodies' systems are biochemical wetware that will never be aptly described using a boolean basis. That is one of the primary problems of society's obsessions with classical notions of gender.

No one is male OR female. We are, every single one of us, a combination of male and female hormones. The more "male" a person is is the result of that balance favoring the male hormones; and vice versa. What humanity is now struggling with is that there are plenty of folks with lots of both or little of either and all kinds of combinations.

Of course, my not being a biochemist means my categorization of hormones into "male" and "female" is, itself, likely to be a poorly booleanized representation of their natures.

We are much more akin to Boltzmann's statistical mechanics description of reality, than to digital logic's boolean description.

> In particular, our peripheral nervous system is capable of absorbing information from the environment at much higher rates, on the order of gigabits/s. This defines a paradox: The vast gulf between the tiny information throughput of human behavior, and the huge information inputs on which the behavior is based. This enormous ratio – about 100,000,000 – remains largely unexplained

The GPU is capable of performing billions of operations per second, yet Cyberpunk barely runs at 60 fps. And there is no paradox at all.

By the way, the brain seems to perform better than a GPU at tasks like image recognition. Probably because it does even more operations per second than the GPU.

Human's can transfer up to 39 bit/s during normal speech, so I highly doubt that it's accurate to describe human "throughput" as being only 10 bit/s.

https://www.science.org/content/article/human-speech-may-hav...

I think the authors are using information theory to inappropriately flatten the complexity of the problem. On one hand we have “bits” of pre-processed sensory measurement data, then on the other hand we have “bits” of post-processed symbolic data: in many cases directly so via human language, but that would also include “the Terran unit moved a short distance” as a compact summary of a bunch of pixels updating in StarCraft. This even extends to the animal examples: the 10 bits/s figure applies to higher-level cognition. The crucial difference is that the sensory bits can be interpreted via the same “algorithm” in a context-independent way, whereas the higher-level cognition bits need their algorithms chosen very carefully (perhaps being modified at runtime).

So I am just not sure why 10 bits/s of symbolic data processing is especially slow in the first place. We don’t have a relevant technological comparison because none of our technology actually processes data in that fashion.

The optical nerve has an information density of around 10 MBit/s (ref https://pmc.ncbi.nlm.nih.gov/articles/PMC1564115/) Concentrating on only the symbolic thinking speed seems to be unnecessarily restrictive..

The authors of the article are smuggling in the assumption that 10bits/s is slow.

It's slow when compared to general computing system that we implemented in silicon substrate.

But this assumption doesn't translate linearly to the brain throughput and the perception of existence.

In my opinion the hypothesis is meaningless.

That is not to say the article is meaningless. Actually being able to measure brain information throughput is amazing. It's only that slowness isn't absolute.

> thinker can access about 220 ≈ 1 million possible items in the few seconds allotted

Huh, no? No one is able to think about million items in a few seconds.

The 20q thinking process involves bringing an incomplete set of abstract categories and asking a question that divides these categories into two halves (binary search). You don't even start from scratch, using previous experience (cache) to reuse whatever worked best the last time.

I see there are some people in the thread that doubt the low bandwidth between conscious thought and the rest of the central nervous system.

Do you also doubt that you're actually living half a second in the past, with the brain compensating for this lag between initial perception and conscious reception of the indirect effects of it?

arxiv: https://arxiv.org/abs/2408.10234

The title is a reference to "The unbearable lightness of being" by Milan Kundera for those unaware.

FWIW the title of this article is a play on the title of Milan Kundera's famous book "The Unbearable Lightness of Being".

There's interesting discussion with the authors here: https://www.alphaxiv.org/abs/2408.10234

I tried to read the article, but celldotcom has a presumably very high bit rate robot that promptly questioned my humanity, so I did the dishes and ate lunch, but that didn't get through somehow as proof.(of humanity) And so my recourse is to then read the coments here, to try and get the gist of the argument,but even well fed, doing 11 of maybe 12 bits per second, there does not seem to be any point in quibling with reality. Maybe after a bit of shock-o-late icecream, (B and G chockolate therapy with esspresso beans added)

We process information at the rate we do because that's what works for adaptation to our environment. That's how all biological things develop. Some things are remnants of old adaptations, but generally speaking, we are the way we are because that's what enables our survival. If it seems "slow", it's because it doesn't need to be faster.

Better questions would be, why are we so weak? Why are we defenseless? Why are we hairless? Why do we move so slow? Most other mammals our size are much better equipped for survival. Obviously other beings "perform better" in many ways. Yet our fleshy, weak, slow, gangly, shivery bodies are suited for thinking, for adapting, for communicating and collaborating. However unoptimal or "slow" these authors think our brains are, they are obviously perfectly capable for what they need to do, seeing as we dominate the entire planet.

In any system design, every decision tends to be a tradeoff. You can gain CPU power, but it will cost you energy, heat, and probably stability or longevity. You often do not know what a "feature" will do until you add it. So it's wiser to only add the features you need to accomplish your immediate goals.

If at some point in the future, our survival is existentially threatened by our lack of "processing power", our brains will adapt. Until then, leave well enough alone.

Why not? Processing an input of 10^9 bits, making sense of all of that, and contrast it against all your existing knowledge have an output speed 10 bits/s? It is not so bad. At least if we were really processing all that information in the same way.

It had to be enough to let us survive, in the context of the challenges we faced through most of our evolution. We took a lot of shortcuts and trims there, that is why we have a system 1 and system 2 in place, with a lot of builtin cognitive biases because of that.

> Why can we only think about one thing at a time?

This sort of reasoning seems to be a symptom of inadequate communication/jargon/diction describing mental faculties. Many times during serious thought there's no discrete "number of thoughts" occuring at all: there's just a hazy mental process that resolves to some result and often many results. This reminds me of the "80% of people have no inner monologue!!!" bullshit that went around recently.

What would it really even mean for our brain to be fast or slow? Relative to what?

Inherent speed does not matter and is ill defined, it only matters relative to the environmental processes we have to react to. We’re already orders of magnitude faster than processes like geology and weather, so it is really only other animals where speed matters, and since we’re all using the same basic hardware, we’re on a relatively level playing field there (except for house flies it seems, lol). Time as we understand it may be as much as a cultural construct as anything else (see, for example Timeless physics). Some neurodivergent people, and people from very different cultures don’t experience anything like what most people in our culture refer to as time.

As for thinking about one thing at a time- I am absolutely certain this is false. Our subconscious operates massively parallel, and we only have one conscious thought at a time. But it had an interrupt system that lets it either act instantly itself in an emergency, and to bring things to consciousness when they are important. I’m sure everyone has had the experience of reacting quickly to an emergency with no conscious thought, or suddenly knowing the solution to a problem you hadn’t even been consciously thinking about.

I skimmed over the paper so sorry if I didn't see it, but isn't the most obvious answer that (at least conscious) human decision making happens at a high level of abstraction? Information output or processing in terms of bits, say the numbers of words in a sentence, isn't equivalent to the meaning or effect that has in the world, which is what matters.

If a general orders an army to go to war that's a few bits of text on a piece of paper, but obviously that bears no relation to the consequence of what that puts into motion in the real world. It's not like we spend most of our days speedcubing or writing and reading the dictionary. We aren't low level data processors even though that might happen unconsciously somewhere in the sensory system, but the conscious mind is a reasoning system.

Even in artificial systems is that visible, from Deepmind on Alpha Zero (https://deepmind.google/discover/blog/alphazero-shedding-new...)

"For each move, AlphaZero searches only a small fraction of the positions considered by traditional chess engines. In Chess, for example, it searches only 60 thousand positions per second in chess, compared to roughly 60 million for Stockfish."

Not too draw to many parallels between the human brain and these systems, but they do obviously share the similarity that higher order conceptual decision making compared to just data processing will result in lower rates of decision making at at least the top level. That's for me what you'd expect to happen, not a paradox.

Found the pre print if you don't have access

https://arxiv.org/abs/2408.10234

I think the discussion on serial vs parallel processing is incomplete in the linked paper, and is one reason I think the 10 bits/s bottleneck is an incomplete or misinterpreted result. Here’s a review with sources on serial processing, https://journalofcognition.org/articles/10.5334/joc.185

> Cognitive psychology has mainly focused on structural and functional limitations of cognitive processes when facing multitasking requirements. Structural limitations assume strict serial processing for at least one processing stage, while functional limitations assume flexible, parallel processing only limited by the number of available resources. Human movement science, on the other hand, emphasizes the plasticity of cognition and training possibilities. As both approaches have provided ample empirical evidence for their views but have predominantly worked in isolation, this example clearly illustrates the need for a more integrative approach to multitasking. A challenge for the contemporary research on multitasking is to bring together the issues of structure, flexibility, and plasticity in human multitasking, offering a new integrative theoretical framework that accounts for this fundamental aspect of human behaviour.

From one of the papers cited by the above reference (Hommel 2020),

> A closer look reveals that the questions being asked in dual-task research are not particularly interesting or realistic, and the answers being given lack mechanistic detail. In fact, present theorizing can be considered mere empirical generalization, which has led to merely labeling processing bottlenecks rather than describing how they operate and how they actually produce the bottleneck.

So, while I applaud the authors on generating buzz and discussion, I think their promising work will benefit from more serious consideration of the underlying neurophysiology.

> More generally, the information throughput of human behavior is about 10 bits/s.

I'm sorry, I just can't take this article seriously. They make a fundamental mistake of encoding and assume that information is discretized into word-sized or action-sized chunks.

A good example is a seemingly discrete activity such as playing a musical instrument, like a guitar. A guitar has frets and strings, a seemingly small number of finite notes it can play. So it would seem a perfect candidate for discretization along the lines of the musical scale. But any guitar player or listener knows that a guitar is not a keyboard or midi synth:

1. The attack velocity and angle of the pick intones aggression and emotion, not just along a few prescribed lines like "an angry or sad or loud or quiet".

2. Timing idiosyncracies like being slightly before or after a beat, or speeding up or slowing down, or even arhythmic; the entire expression of a piece of music is changed by subtleties in phrasing.

3. Microbends. The analog nature of strings cannot be hidden entirely behind frets. Differences in the amount of pressure, how close to the fret the fingers are, and slight bending of the strings, intentional or unintentional, static or dynamic, change the pitch of the note.

4. Non-striking sounds like the amount of palming, pick scraping, tapping, and sympathetic vibrations.

Of course there are lots of other things. All of these things make the difference between a master guitar player, say Hendrix, and someone just playing the same notes.

And yes of course we can consider the encoding of the audio coming out of the guitar to be information--at a much higher bitrate, but what about the facial expressions, body language, etc? There are tons of channels coming off a musician, particularly live performances.

This entire article just misses these in picking a quantized encoding of information that of course has a low bitrate. In short, they are missing bazillions of channels, not the least of which is expression and timing.

You can compress 10Gbit/s down to 10bit/s but then you can't do the reverse operation... Yeah, we know.

To me it's like saying "I've summarised this book to 5 words, so why would you even need a few days to read it entirely?".

Definitely a neat article insofar as it summarizes many observations about task related “bandwidth” of human output.

Summarizing other comments - calculating bandwidth in a task-specific way neglects metacognition: the Rubik’s cube solver can rapidly recover if someone shoves them when their blindfold is on; the memory champion can notice a pie being aimed at their face and duck. I think that there’s still something interesting around our inability to multitask conscious behaviors, but the overarching conclusion is flawed.

I’m interested in the phenomenon that no matter how messed up (or absent) my sensory input becomes it doesn’t seem to impact my “inner observer”. The only thing impacting it is sleep or loss of consciousness, yet even then there can be moments during these states that this observer can become lucid.

It’s like the science of the soul and one notable implication is dementia, when people become unable to function and recognize things, I wonder if that “inner observer” is still just as in-tact as it is on a huge dose of LSD?

Free access link https://arxiv.org/pdf/2408.10234

Seems like onsciousness is the bottleneck. It has to integrate over all the perceptions. Of course this will be slower!

The answer is that we defined second after the time period it takes for us to think about something, if our thinking was faster the second would also be faster and we would still feel like we only think about about one thing per second.

> Plausible explanations exist for the large neuron numbers in the outer brain, but not for the inner brain, and we propose new research directions to remedy this.

Going on a limb here, but perhaps we shouldn’t modify biological composition of the human brain.

It seems the authors conflate a problem bring easy to state with little processing power being needed to solve it. This obviously isn't true, very complex mathematical problems can be stated in very few bits. Human interactions often can be extremely complex, even though they are relatively slow.

Reading a text isn't about matching symbols to words. It is about taking these words and putting them into a social context, potentially even doubting their content or imagining the inner world of the author. Obviously that is what the "inner" brain (which existence seems very dubious to me) has to do.

I see absolutely no paradox at all.

"Why can we only think about one thing at a time?" - this is not true? There is a difference between thinking and conscious thinking.

I prefer to think of the inherent slowness of my organic brain as a leisurely stroll through time. I’m taking the scenic route though life.

Interesting summary, where can I get the full text?

Related question: what's the physical/biological difference between conscious and unconscious?

Our awareness of time seem arbitrary. If feels time went by fast/slow, how does that work anyways?

So the human brain is a great operating system that is desperately missing a good text editor...

10"very-ultra-extreme-high-level"Bit/s (maybe)

10 baud, perhaps. Definitely not 10 bits/s!

This is such a bad paper. Almost all calculations and equations look like some back of envelope calculation. A decent researcher would have provided some tests to their hypotheses.

Thinking is emulated -- therefore much slower.

Ask stupid questions, receive stupid answers.

there's probably lots of local computation

and i think it's a mistake to simplify it all down to just one substrate or receptor

It seems this analysis is incorrectly assuming a serial communication mode for neuronal transmission, which isn’t what happens.

On average, people can read at least 200 WPM, but much higher at the top end. This is orders of magnitude higher than 10 bps.

perhaps the 10 bits/s is throttled at the simulation level ;-)

"It is by will alone I set my mind in motion. It is by the juice of Sapho that thoughts acquire speed, the lips acquire stains, the stains become a warning. It is by will alone I set my mind in motion."

The back of the envelope computations are shockingly shallow and meaningless.

(20 Questions, from the intro) Trying to think of a thing for the game is not a search over a set of known things. Just saying the possibility set has size 2^N doesn't mean that choosing something in the set consists of processing the set. But even if that were the case, and if you do consider each of 2^N options, the consideration process itself is not trivial and probably varies wildly.

(English typing) Touch typists do not (only) simply convert an existing/known string to a sequence of hand actions by mapping character to action. There are whole words and sequences that become units/tokens from the standpoint of muscle memory and processing (this will be relevant to the rubik's cube topic as well). When i type, there's a sort of planning and queueing of actions, but also there's monitoring of actions that allows fast error correction with pressing delete a number of times or holding it and costly determining when the error has been reached, and resuming afterward. Of course the process likely varies from person to person, but there's such a host of other things going on that should count as part of the information processed in this simple behavior that the example and numbers used in the paper for it are utterly useless even as estimates.

(Rubik's cube blind speed solving) Again we see reference to the entire possibility space (from the perspective of possible configurations of the puzzle). But solvers do not identify the configuration they encounter with reference to the space, nor do they search the space for it. They look for patterns and ignore what they cannot use for the strategy they have practiced. The cuber often does not commit to memory the whole configuration, but will often convert it to a custom and bespoke mnemonic. It's just utter nonsense to refer to the number of possible configurations, it has nothing directly to do with what the human is doing.

If I memorize a 30 word passage, i have not "processed the set of possible 30 word passages".

These types of articles are so fundamentally flawed... it beggars belief. Why not ask the opposite question: if bandwidth works the way they describe, why can't H100 GPUs (3TB/s bandwidth) perform sensorimotor tasks 24 trillion times faster than a human? (Spoiler alert: they can not).

<s> Could it be... there is a bit of a straw man argument here? About how much information it actually takes to input and output a complete sensorimotor task? I dare say! </s

Bits are a unit of measurement we use in relation to computers. Humans are not computers. Do not use bits to measure anything in relation to humans. Stop thinking of humans as computers. It's dehumanizing, unhealthy, and a very narrow way to think of people. Using a computer model for humans is useless at best and misleading at worst.

It's 1⁢0^9 bits/s. Your title is wrong.

I read somewhere that the eye transmits 10 Million bits per second to the brain. I think all of this is a matter of perspective.

The 10 bit/s processing rate doesn't explain why a human talks better than a LLM that consumed terabytes of data traffic during learning.