Brain Computation Is Organized via Power-of-Two-Based Permutation Logic | Hacker News Reader

Brain Computation Is Organized via Power-of-Two-Based Permutation Logic

journal.frontiersin.org

134 pointsaburan2854 comments

There are only a few organizational principles that allow the number of groups (K) to scale beyond the number of neurons (N).

Polychronization (http://www.izhikevich.org/publications/spnet.pdf).

This is e.g. different from synfire chains in which the number of groups equals the number of neurons. I don't remember much about the capacity analysis of standard reservoir computing methods (ESN, LSM, ELM), but I thought it was also limited to N.

I'm looking forward to more research in this direction! We as humans build up such long-term memories. This must be a driving factor in AI. A lot of prediction tasks hinge on being able to handle this huge temporal scale.

Currently I'm trying to see if I can formulate chains of transition matrices in MCMC that allow simultaneous search on multiple scales. I feel like the math from multiple angles is converging, exciting! We're gonna solve this in our lifetimes!

Is ELM reservoir computing?

Yes, if you consider the most salient part of reservoir computing method a random projection matrix that is quenched (kept fixed).

http://www.nature.com/articles/srep14945

I spent way too long staring at figure 1, before realizing that they're not using the word "permutation" in the way that people with exposure to CS or discrete math would use it. They're assigning a "neural clique" (possibly not exactly a clique in the graph theory sense) to each subset in the powerset of inputs, and then ignoring the empty set. _Now_ 2^i - 1 makes sense; ordering does not matter to the authors.

It seems like you are saying that the headline is nonsense because the authors of the article are using non-standard meanings for their words.

Any system with optional components has a factor of 2^n in its count of configurations -- each optional component might be absent. That's an interesting fact of math, not of any particular system.

I am not saying it is nonsense -- just that it was hard for me to understand their claim, in part due to a background that I expect many other readers here share.

It seems that by "permutation" they mean what a mathematician calls a "subset", or a computer scientists calls a "bitmask". Essentially they found lots of evidence that animal brains implement demultiplexers?

This sets off my BS detector.

The approach is to make a "thought experiment" about how neural computation might work mathematically, then look for evidence that the guess is correct. Their thought experiment makes them guess powers of 2 are special, that they should expect to see organization at sizes 1,3,7,15.

But when they analyze data, I don't see any comparison saying that 7 is really more prevalent than 6 or 8, or 15 is more prevalent than 14 or 16. Indeed 15 seems to be the largest they looked at.

The whole numerology theory driving the headline looks like unlikely, unsupported nonsense to me!

This seems pretty hand wavy. I'm not seeing any actionable insights from this.

For real though. We already knew that neurons can do binary logic, they either fire or they don't and neurons higher up the chain fire based on converging inputs.

Also what do those plots actually 'show'. The authors claim repeatedly that the plots show the existence of 15 clusters, but without stating p values relating to any sort of classification for many of the plots.

Would anyone here be able to offer some book suggestions for tackling AI from the neuroscience/ biological perspective, as opposed to the computer science/ computational perspective? I want to expand my horizons.

The following suggestion is not about tackling AI from neuroscience/biological perspective, but one that critiques the computational view of mind/brain - which provides only a limited picture.

See the work of prof. Walter Freeman: https://en.wikipedia.org/wiki/Walter_Jackson_Freeman_III

Esp: Reclaiming Cognition - Ed. Nunez and Freeman.

Also see the work of Hubert Dreyfus.

These works will help you figure out what 'tackling AI from a neuroscience' perspective could actually mean.

I would suggest "Principles of Neural Design", a book that tries to explain why things might be engineered the way they are in biology:

https://mitpress.mit.edu/neuraldesign%20

"explains" and "might" are kind of at odds, here

https://en.wikipedia.org/wiki/On_Intelligence

I don't think anyone serious buys this version of the GUT. You'd be spending your time better reading Randy O'Reilly's work (CU Boulder). Or: https://en.m.wikipedia.org/wiki/Rethinking_Innateness

Or Gregory Bateson

Or Josh Tenenbaum (MIT)

Wouldn't that be just "I", and the whole field of cognitive neuroscience?

Wai H. Tsang:

http://www.academia.edu/27961545/The_Fractal_Brain_Theory_-_...

Except for the Dopaminergic neurons, among other "subsystems".

That is explicitly mentioned in the article.

This simple mathematical logic can account for brain computation across the entire evolutionary spectrum, ranging from the simplest neural networks to the most complex.

That is, further research will investigate how e.g. a mouse turns in a particular direction in a maze before investigating how high-performing humans perform amazing mental feats. It may all be the same at some level, but we have to walk before we can run.

Some questions:

We act like stimuli are exact things but the brain is all about abstraction. In fact, a certain amount of entropy/information enters your beain every day from vision, smell, touch, etc. Its not stored in full fidelity. There arent enough nuerons or energy for that. So what is lost? Well...whatever isnt abstractable, right? You celebrate your birthday. You remember blowing out the cake in pristine vision. But how much of that 'movie reel' inside your head is actually a superposition of abstracted reality vs actual observed reality?

All Im asking is, is it even worth it to think of a human being in terms of absolute information when their memories are so sparse from reality. We remember whats important. And the things that are important, weve remembered to remember. But ultimately, the actual holding power of the brain is quite small. The brain is a master of deep abstraction from sparsity.

Is it like a quantum system? Where the superposition values of one subcortical system are flattened by another? So that a whole brain belief/memory is a function of the fuzzyness of each subsystem being exacted/wave function flattened by the heuristic correlation/connection between all the shitty data? I think we need to start thinking about sparsity. The universe is sparse when it comes to using 3d space. Holographic principle says physics is so symmetrical that we sparsely use 3d space. Such that our entire universe could take place on 2d space. Max Tegmark gave a talk about how deep learning works so well because of the sparsity of faces or music or voices, the overwhelming symmetry/redundance in physics and nature. And transitively, humans.

Lossy compression combined with abstraction mechanism might be more a more appropriate way to describe it.

What gets processed and enters consciousness only a small percentage of the actual raw input from sensory "devices".

What get's stored is even less than that, is highly inaccurate and degrades quickly.

Abstraction comes in at finding patterns in the lossy data, and filling in the gaps in the data to form a coherent perception/memory.

My take is some kind of deduplication. Similar input gets overlayed, while novel inputs gets a new "slot". Thus we remember more about our childhoods because they are filled with novel experiences, while as we grow older time seems to fly because one day merge with all the others.

I don't know about it being a quantum system, but from a psychological standpoint our memories are definitely shaped by the way we perceive and process an event. There was an article posted on HN recently about why we don't remember being babies - some think that language is required for forming memories. The way a shared memory is remembered and re-told by other people (i.e. parents) can also affect our own memory of it.

As a person suffering from social anxiety, my brain tends to memorise events by focusing on small, irrelevant details - which leads to negative self-talk, i.e. "was this thing I said embarrassing? could I have done this differently?, etc."

The way we perceive the world and store our memories is definitely different from one person to another. As a bilingual person, I think some of those differences may indeed be due to a person's native language.

If language is required for forming memories, surely babies that have learned sign-language would remember something, no?

https://en.wikipedia.org/wiki/Hyperthymesia

As a bilingual person, do you dream in both languages, or one particular language?

In my case after living in another country for about 3-6 months I've started having dreams in new language. It came more less together with having inner dialogue in the head in new language. New one replaced the old one.

That just means your language is part of your memory, not that it is necessary for it to function.

> So what is lost? Well...whatever isnt abstractable, right?

Raises a most curious question.. what ever isnt abstractable?

> what ever isnt abstractable?

If you are interested in this kind of question, read up on Kolmogorov complexity:

https://en.wikipedia.org/wiki/Kolmogorov_complexity

We're just systems that transport our genes into the next generation. Our perception exists mostly to help us find food, avoid dangers and reproduce. Of course we lose a lot of information, because it is irrelevant to us.

I use my perception to do much more, most of the time. All evidence points to humans being a good few steps above other animals for whom those three activities are not only necessary but almost sufficient.

Humans might be above animals, but I suspect not by much. Who's to say what minor event in evolution prevents dolphins or AI in becoming sentient.

Opposable thumbs, most likely.

Fortunately, evolution selected for intellect even though it might have initially have been past a minimum fitness/energy valley. Id encourage you to rethink this simplistic view that ignores the general tool that developed in order to handle more random and abstract specific solutions to threats.

There is no proof that smarter brain is what was targeted by evolution. It could be that the size of human brain was either purely accidental consequence of other factor or at least was driven by the needs like balancing when walking/running on 2 legs.

There's no such thing as "targeted by evolution". There's passing on genes and not passing on genes. This tends to happen with individuals that are better at survival (or that procreate more). Modifications to genes to create an individual better at passing on genes are essentially random.

A simple reality check:

Considering the large energy needs of the brain, there is an evolutionary pressure to make it smaller... so something pushed it the other way.

Compared with apes humans much more efficient at digesting of starch. However to get enough protein and micronutrients from starchy root vegetables one has to consume much more in calories compared with vegetables. That effectively provided "free" energy to keep bigger brain. Of cause this is rather speculative, but it just emphasises that we really do not now why humans develop bigger brain.

Another abundant energy source is fish protein, which always existed but our ancestors species is the only species that went nuts over fishing. Its pretty easy for hominids or scavengers in general to get fish protein at a net loss, or in very small quantities, but once you have a big enough brain our species really got into eating fish starting about 100Kyrs ago.

Apparently there is evidence you need a pretty big brain and speech abilities to run a net energy positive fishing village. A neanderthal should be able to catch fish, just not efficiently enough to live off a fishing culture. Their trash piles have all kinds of bones, just not fish bones. Ours are/were full of fish bones, anywhere there's fish.

Fishing might have been "the niche" that led to our species... starting with older hominids who were better than average at gathering fish due to some local geologic peculiarity (the perfect river to catch salmon by hand occasionally or whatever), a zillion generation later of ever improving therefore ever fatter fisherman and we got a prehistoric Captain Ahab filling the tribe trash piles with fish bones and the non-homo sapiens cousins can't fish compared to us and die off because we're fat from fish and they aren't.

Large amounts of starch would seem to require agriculture which is pretty recent compared to fishing.

@VLM, ice age ancestors needed big game. Small fish are a joke for sustained energy and nourishment, maybe as a condiment to the main meal of mammoth or aurochs.

Our "ancestors" really thrived for the most part of the very mega-fauna we helped extinguish.

> Their trash piles have all kinds of bones, just not fish bones. Ours are/were full of fish bones, anywhere there's fish.

Your sources seem to wildly differ from my sources regarding the bulk of ancestral homo sapiens evolution during the ice ages. Feel free to share them.

Human ancestors got access to fire at least one million years ago. And with it it is trivial to cook starchy roots by baking in ashes.

As for the need for agriculture consider that single modern day hunter gatherer in Amazon forest can within few hours get enough starchy vegetables in a forest to feed at least 10 persons.

Human ancestors got access to fire at least one million years ago. And with it it is trivial to cook starchy roots by baking in ashes.

As for the need for agriculture consider that single modern day hunter gatherer in Amazon forest can within few hours get enough starchy vegetables in a forest to feed at least 10 persons.

> Of cause this is rather speculative

Insanely speculative. Homo sapiens brains were largest 90k years ago, stayed there until 60k years ago, shrunk a little until ~10-15k years ago, then began shrinking markedly.

This coincides with a similarly slow transition from ice-age ~99%-carnivorous hunters (biggest brain), to hunter-gatherers (a little smaller, still big), to agriculture (begin massive shrinkage).

How do you get "starches" in any calorically significant quantity?

Oh yeah, agriculture is the name of the game.

But even all THIS "is rather speculative" as well: after all, bigger brain cannot just be equated to "smarter brain", either. Maybe starch allowed the brain to retain intelligence more efficiently and thus to shrink! Personally, I don't believe it, but frankly until we get a frozen ice-age homo brain, we won't have any way of knowing. Even then, it's just better-informed speculation.

Fun idea. I find that rather unlikely. Some points.

Are there any evidence for a period without meat in the diet, to start with? Considering that teeth are hardy, this will probably be testable in our lifetime.

I've rather seen claims that the brains were larger before civilization, following the hypothesis that it is hard to live as a hunter gatherer. (And then it was with lots of meat in the diet.)

We do know that calories have been a limiting factor for most life, generally also including humans. iirc, 20% of the energy need goes to the brain. The brain size would vary a lot, if it really didn't make a big difference.

Consider that the evolved tendencies for getting diabetes are there because it made it easier to handle periods of hunger. So sure, that might be much later, but...

And also, eating those starches needed cooking for humans, afaik? Which needs some brain power (there was not time to evolve instincts). We lost a lot in the jaws and teeth to get the brains. (Couldn't have both.)

Then, there was news just a week or two ago about a new result -- humans put "building capital" into our brains rather than our muscles, which is why most monkeys while large as children can tear a human's arm off and beat our head in with it... Brains are expensive.

I might add that there have to be a hard evolutionary pressure to build complex organs, like the extra stuff in the human brain. It is hardly just more neurons added to earlier monkeys (evolution is hill climbing, so you will find simpler/different analogues in other primates, but it isn't just adding some volume and e.g. <miracle> language circuits :-) ).

And so on. This list could probably be longer. :-)

There's another point: big heads (whether they contain a big brain or not) are selected against, as they make birth a much more dangerous business that it would be otherwise.

https://www.academia.edu/1276752/The_causes_of_brain_enlarge... didcusses that bigger brain could have evolved as it was easier to keep brain functional under intense heat.

Evolution do not "target" jack all. If gene A survive over gene B it simply means that A is a better fit for the present local conditions.

Antromorphism is a common figure of speech, just like your "it means".

Andy Clark - Perceiving as Predicting https://youtu.be/05P41FQlgjI?t=703

To answer some of the questions:

"But how much of that 'movie reel' inside your head is actually a superposition of abstracted reality vs actual observed reality? All Im asking is, is it even worth it to think of a human being in terms of absolute information when their memories are so sparse from reality. We remember whats important. And the things that are important, weve remembered to remember. But ultimately, the actual holding power of the brain is quite small. The brain is a master of deep abstraction from sparsity. "

First, not everyone has a 'movie reel' type of memory [0]. ~2% of the adult human population 'just remember' and have no 'mind's eye' or 'movie' playing in their heads [1]. We also know of many savants[2] that have 'photographic' memory and can perfectly recall images and places, but many of them struggle with other tasks in life. Also, some savants can acquire their incredible abilities after brain trauma late in life. So, the answer to the questions is that it depends greatly on who you ask and when you ask them. Your brain is astoundingly strange and continues to be so at nearly every age.

"Is it like a quantum system? Where the superposition values of one subcortical system are flattened by another? So that a whole brain belief/memory is a function of the fuzzyness of each subsystem being exacted/wave function flattened by the heuristic correlation/connection between all the shitty data?"

Your brain is decidedly non-quantum. Very basically, as we understand it today, the way memory works is via synapses and the modulation of how 'strong' they are [3]. The 'stronger' a synaptic bond to a neuron, the better the transmission of 'signal' over other competing synapses to a neuron. Nature does some pretty crazy things to keep that 'idea' going, like the calyx of Held [4]. I'll repeat that, in many brain areas so far studied, but not all, your brain uses a competition system to transmit signal.

I don't really understand the next questions. What do you mean by 'flattened'?

On your points about the universe being 'flat', which I assume is different than the flattened statements you talked about, you have to remember what the brain is 'for'. Our brains eat up ~20-25% of our daily food intake [5]. That is a LOT for mega-fauna mammals. Nature has deemed that this increased usage of the food is ok though, as it helps us get more food than what we spent. Obviously, with the obesity epidemic, we are kinda past this point, but we have a big brain that helps us eat. That is one idea about why the brain is so complex, that it helps us eat more food.

Another is sexual competition. Large apes like Humans, Gorillas, and Ourangs have very complicated dominance and social structures. The reasons for this are another thesis, but it ends up helping us survive and have babies. If you have a 'theory of mind' about your troop-mates, then you can anticipate their actions and maybe be dominant and make the babies. That takes a LOT of 'processing power' and, I don't think, is a 2-D problem. Interestingly, this related highly to the eyes and knowing and guessing what a rival is doing, hence the glasses they give out to zoo patrons these days [6]. Also, dogs are the only non-primate that can do this, again a possibly co-evolution related mechanism.

Basically, your brain is 'for' 3 things. Other evo-devo scientists say it in this joking way:

You are a mammal and you encounter some thing that is novel and you have not seen before. The analysis schema is as such: 1) Can it eat me? A: No. 2) Can I eat it? A: No. 3) Can I have sex with it? A: No. 4) Ignore it forever.

The brain is 'for' trying to go through that schema as fast as possible and with as high a degree of fidelity as possible. Again, jokingly.

[0]https://en.wikipedia.org/wiki/Mind_projection_fallacy

[1]http://www.bbc.co.uk/news/health-34039054

[2]https://en.wikipedia.org/wiki/Savant_syndrome

[3]https://en.wikipedia.org/wiki/Synaptic_plasticity

[4]https://en.wikipedia.org/wiki/Calyx_of_Held

[5]https://www.sharecare.com/health/calories/brain-calories-at-...

[6]https://boingboing.net/2009/04/17/gorilla-viewing-glas.html

Why that pdf in the title? The page is a long html article.

Reply on news.ycombinator.com