The cortex is a neural network of neural networks
medium.com
medium.com
Neural networks of neural networks doesn’t begin to describe it. We haven’t even scratched the tip of the tip of the iceberg in understanding this stuff.
In fact, I think you're probably dead wrong. Certainly hormone interactions and feedback loops in the body have some consequence on an organism, but these things take time. Lets say that some feedback loop is extremely fast, say 10 seconds. It seems reasonable to assume that you can't have a complex feedback loop with the body much faster than that, just because it takes time for chemicals to physically move. Cognition is much, much faster than that. There are many situations where you can learn, and then apply that learning in less than a second. Now if we were instead talking about what it takes to build a complex organism that can satisfy it's own needs, then certainly these complex hormonal feedback loops are essential. You can't have an organism survive when it doesn't look for food when it's body is out of fuel or keeps eating poisonous berries because it's body can't send the reasonable feedback to it's brain, and you can forget about dropping everything and competing for mates at the correct time without some kind of behavior modifying hormonal signals!
I would say that your assertion is not significantly different from saying "you can't understand the workings of the brain without considering the complex interactions of the brain with the lungs, since without oxygen the brain can't work". So while I agree that the brain exists as part of the body, I don't see why you would automatically assume that the hormonal feedback loops with the body are at all necessary for cognition rather than a way to tell the brain when it's necessary to find food, when it's adaptive to conserve energy and be lethargic, or what have you.
My point is that we may or may not find that there is some critical ingredient of cognition hiding in this place or that, and you can't very well tell us where that will be if nobody knows if it's there.
My point was that we can't just determine the typical time constant for a hormone's effect and run home with it since there are chemicals which have significantly lower time constants and some of them act as endocrine hormones as well.
Furthermore, an immediate feedback loop isn't the only possible way the body could play a role in cognition, though I suppose the OP is aware of this and was talking about moment-by-moment interaction with the body on purpose.
So yes I think we agree that it’s all more complicated than anyone can even comprehend just yet.
It's like the parable of the Drunken man looking for his lost keys under the street-lamp. He only looks under the lamp for his keys because that's the only place the light is.
Similarly, in neuroscience, we only have a few tools to look at the brain. By some miracle, the brain happens to be electrically active to a degree that we can shove wires into it and pick up signals. So, dutifully, grad students across the world have been shoving needles into brains and trying to pick up signals and then tell advisers what that means.
Another miracle is that blood has a ferromagnet in it and the brain has a fair bit of blood in it. So, after a lot of trial and error, we've been able to build very complicated fMRI machines that can give grad students an idea of the usage of blood in the brain. This is then correlated with other measurements to find out what is happening in the brain.
Recently, we've been able to combine two different little miracles together. One is opto-genetics, a little serendipity we pulled out of the chromatic hot-pools in Yellowstone. Essentially, when you shine light on a little bacteria, it's cell wall opens up and you can squeak certain ions through the hole. The other little miracle is CRISPR-CAS9. Here, you can again use some tricks bacteria have made up to more easily copy-paste in the genetic code. Using these tricks and a lot of help from post-docs, yet more grad students have been able to report results in lab meetings to their advisers.
More techniques exist, of course, but my point is that the vast majority of neuroscience is still in the dark, away from the drunken man searching for his keys in the light.
Look at astrocytes. These guys make up ~50% of the brain. Very recent research says that they play something of a role in the synapse, moving NMDA receptors around the cleft, recycling, phospho-tagging recpetor, etc. It's super recent work. But also super important. Before, we thought that only the neurons were involved in the synapse, but now we have good evidence that other, action-potential lacking, cells are also deeply involved.
That vein of research is also really hard to do and preform experiments in. As such, most grad students are wise not to do their work in that kinda lab. It's hard and has a high potential of failure, and therefore no degree.
So, wait and see. Our understanding of the brain is still very much in it's infancy. We've not really even got a good accounting of all the cell types and connections in the brain yet.
What we are really building are pretty good pattern recognition machines, often with surprising capabilities. But we overreach when we equate that with intelligence.
At least it's useful to discuss the ethics due to buzzwords, media hype and industry demands before we are actually able to produce something truly artificially intelligent.
Until then, we continue to add DL algorithms to our Madam Tousseau-esque collection of artificial intelligence.
Surely there’s going to be some sort of crash when they collectively realise that’s not the case.
Even a few years ago, I worked as a student at a Big4 and these firms ran out for "technology consulting" and told their clients about AI this, AI that and it's gonna be robots taking it all over. Of course those were advertisements, ruses even, for getting audit & tax customers via the "compliance" angle (e.g. cloud compliance laws).
But anyway, their is a true craze about AI and autonomous driving which holistically ignores all the major road blocks that we are facing now. It's buzzwords and marketing all over. It's the Gluten-intolerance of technology.
According to general public media we are 2-5 years away from level 5 autonomy. Ask people at Waymo about the issues they face...
Domain-specific AI is often already superhuman in performance, once trained. What we lack seems to be: (1) generalisation, as a trained AI is often useless in separate tasks; (2) efficient training, as AI take far more data to reach the same performance as a human (I have seen suggestions this is related to (1)); and (3) any idea at all what counts as self-awareness/conscious/qualia/etc., which may not be important from a performance point of view, but very much influences how people regard the AI and their future potential.
What is really the case is that some AI systems (in particular, deep neural net classifiers) have superhuman accuracy and that again only in classification tasks.
Edit: I'm not being contrary for the sake of it. I think there is a very useful insight to draw from the success of deep neural network classifiers: that it is possible to perform classification without any kind of understanding of the objects, or their classes, at all. In the past, AI researchers operated under the assumption that reasoning and inference would be required to perform this task, but we know now that such abilities are not necessary if the task is only to classify objects. It's also clear that in some cases dumb classification can replace reasoning- as long as a reasoning problem can be reduced to dumb classification.
However- humans clearly have the ability to reason and draw inferences (regardless of how often we do that successfully). There must be some explanation for this. Why do we have an intelligence that goes beyond simple object recognition? What is the point of having a broad intelligence? It must mean that there is more to intelligence than tasks that can be reduced to classification. So reducing "performance" to "accuracy" (of classification) risks fudging an important difference between the "superhuman" abilities of machines and human abilities.
In the end, the question is what are humans good at and why can't machines do the same things, even though they can beat us roundly in other tasks? What is the difference between tasks that are easy for machines and tasks that are easy for humans?
However, I think you’re making a distinction without a difference going between classification accuracy and performance. That said, I am running on 1.5 hours less sleep than I need today — the only other important metric I can think of right now is the kind of skill which humans book-learn and which get implemented on a computer as an explicit and deliberate algorithm instead of being learned, and I simultaneously don’t count those as AI and think machines have beaten us for decades in that domain.
I do have one other question though: do you regard Alpha(Go|Zero|Star) as nothing more than classification?
I believe in AlphaZero etc the deep neural net component was used to identify moves with a high probability of leading to a winning board position. That's a good example of a problem we used to think would require inference or reasonging but that can, after all, be solved by classification.
Although that's not to say that the same problems can't be solved by inference, plus powerful computation. That remains to be seen.
Edit: Maybe I shouldn't have called it "dumb" classification; that sounds dismissive. There's no doubt that neural net classifiers are impressive in what they do. I mean it to say that they have no understading of their domain, or ability to reason etc.
I don't think we can say for sure that there is more, though. If and when we finally understand how it all works, I wouldn't be completely surprised to find that it is just many layers of pattern matching and associated feedback loops.
We just don't know enough yet.
AI systems are better than humans at making decisions. AI systems can scale to millions of decisions per second and beyond. AI systems don't need to spend 3/4 or their time sleeping or relaxing. AI systems don't need 20 years of training to become experts in their fields, they can be replicated almost instantly.
While the scope of decisions the AIs can do is somewhat limited at the moment, the trend is clear: in the not so distant future AIs will make all economically relevant decisions.
I don't doubt that, just doubt that they will make unanimously good decisions, as expected of a superhuman system. AI systems are yet to be "intelligent" which makes half of their name a PR gag.
Pattern recognition is simply not equivalent to intelligence. It is certainly a pillar of intelligence, however.
> AI systems are better than humans at making decisions
Are they? Dermatology, transportation, face recognition, NLP... in all these fields we are yet to reach human-like performance outside of ultra-specific tasks. There is a difference between recognizing a human face every time, even if obscured by sunglasses or half hidden behind a pint of beer, and recognizing 100.000 human faces per second with an error rate of 8% because of shadows.
Decent performance can be reached when you train AI to do one hyper-specific thing under just the right circumstances with mountains of data which need to be prepared just right. Otherwise you end up with bias and other issues. I wonder how AlphaGo would have performed if it was required to suddenly switch to Mahjong or cooking but I suppose that wasn't its purpose.
Please also mail me a link for systems that prepare training data to automatically and reliably prevent bias and other major issues that lead to malperforming systems.
Demonstrations optimized for PR, AI companies which employ humans as AI-pretenders, and catastrophically failing systems are the reality of AI today. I don't doubt that we will improve but superhuman AI requires more than mere linear improvement from where we are right now.
If you think gene expression within neurons as small RNN, it's neural networks all the way down. RNN within neuron reacts to hormones and chemical signals altering the functioning of individual neurons. There is also feedback loop to other direction.
RNN nodes represent genes and connections between them are regulatory feedback loops in gene expression.
To anyone interested in this, I would highly recommend picking up a copy of “Principles of Neural Design”. It’s a look at how and why brains operate the way they do, from thermodynamic and information theory axioms up. A lot of the biology went a good bit over my head, but the authors do a great job of developing a set of hardware-agnostic principles that all brains follow.
The problem is that the biology of the brain is incredibly complicated. Dynamic instability of microtubules, contribution of extracellular factors, neuron biochemistry, etc etc... Sure, we've extracted signal to reconstruct primitive visual features from the basal ganglia of mammals, but that's a long way off from what we'd need for humans. (I can't easily cite atm, but I'll be happy to come back and put in references if desired.)
The other big issue is that it's quite invasive to get data out of this system. I imagine we'll be making progress on human cloning and artificial organs long before we crack this nut simply because of how disruptive and insufficient current techniques are.
All that said, I'm pretty sure we're all going to die without any archival backup of our brain-encoded memories. Progress will be made, but not in time for us.
We learned how to make airplane wings from the shape of a bird's wing. Of course we should not model our artificial wings so closely as to make a plane with wings that flap. But there was still plenty of stuff to learn by asking the question "why does a bird fly and my contraption doesn't?"
That kind of thing happens all the time in aerodynamics, fluid dynamics, mechanics, etc — precisely because evolution is a pretty good optimization function, and so “natural” solutions can often be very close to optimal, but using hard-to-discover quirks of physics.
Each species' death is millions of years of labwork trashed
A species of spider nailing down how to live on a particular type of rock on a particular island, in a very particular environment over millions of years, is simply not articulate enough "lab work".
Which is of course not an argument for killing off species. But it's an argument against approaching that moral question from such utilitarian perspective. You might easily end up with results you don't like, once you do the cost/benefit analysis in a less hand-wavy manner.
javascript:(function()%7B(function%20()%20%7Bvar%20i%2C%20elements%20%3D%20document.querySelectorAll('body%20*')%3Bfor%20(i%20%3D%200%3B%20i%20%3C%20elements.length%3B%20i%2B%2B)%20%7Bif%20(getComputedStyle(elements%5Bi%5D).position%20%3D%3D%3D%20'fixed')%20%7Belements%5Bi%5D.parentNode.removeChild(elements%5Bi%5D)%3B%7D%7D%7D)()%7D)()https://chrome.google.com/webstore/detail/make-medium-readab...
While I appreciate the article trying to actually understand what's really going on in neural networks, let's not make unnecessary dumbed-down assumptions. At least the subtitle of the article is actually correct. The main title is sensationalist "...17 billion computers!!".
"Computer" is an observer-relative term. There is no physical property of a system which makes it a computer. A "digital computer" is just a tool made of silicon which we use to aid computation (a goal we have). There are many tools (from an abacus to a waterfall) that we can use to aid in computation.
"Computation" isn't anything other than a goal we have. To interpret the brain as engaged in it carries no information and says nothing explanatory. The sense in which a brain is a computer is the same sense in which everything is: a physical system whose state evolution can be used to aid in computation (but isnt: no one uses brains to compute).
A rock and a waterfall perform computations in a theoretical sense if you look at them right, but there's also a different sense in which a bunch of op-amps in a circuit are a computer, a bunch of hydraulic logic gates are a computer, and a Raspberry Pi is a computer, but a rock isn't a computer. In that sense, human and animal brains are computers too.
(I'm before my morning coffee so I apologize for not being able to name "that sense" properly.)
I chose waterfall by symmetry with a hydraulic computer. A waterfall is just what we call a hydraulic computer when we're not using it for computation.
I still only see an observer-relative distinction (ie., in how it is used). One can "juggle rocks" to compute.
You've said something about "inputs" and "useless when removed". Both of these are again observer-relative. Useless gives it away.
An animal isn't "useless" when its brain is removed, because it isn't our use of it which makes it alive.
Computational neuroscience could be written off by your statement that the brain is not a computer, but perhaps soften your stance and accept that that allows for applying tools from computer science to ask questions, just as physicists do.
Computational metaphors arent explanatory, they're illusory sorts of explanations (like narrative) which "satisfy" without providing a causal model (ie., a scientific explanation).
I'm not convinced they have been helpful, and mostly end up giving deeply mistaken impressions about the nature of digital computers -- rather than helpful impressions about the nature of brains.
The key is to explain using layman's terms without making false claims while trying to simplify the subject.
When people call a thing a 'computer' they mean that it can run (potentially complex) algorithms to produce outputs based on inputs, in many cases it would also be assumed that it has memory, and it would not have much in the way of direct physical interactions with the world (although it might be part of systems that have such interactions). Based on these notions, it seems very valid to interpret brains as some kind of computer, made out of cells instead of metal and silicon, and formed by evolutionary pressure instead of human engineering.
It isnt valid to interpret brains this way. An "algorithm" isnt something a physical system can "run". These are observer-relative properties a system instantiates because of our use of it.
A brain is a physical system and our model of it should be a causal model in terms of its intrinsic properties that do not "disappear" when not in use.
A digital computer is an oscillating electrical field across a piece of silicon whose oscillations "correspond" to steps in an "algorithm" but only to an observer who imposes those correspondances. The program "1 + 1" on a digital computer is some state transitions of the cpu's electrical state -- it is an algorithm only when we impose an meaning on those oscillations that they do not themselves have.
There is no physical distinction in the "waterfall computer" and "digital computer" that bares on them being computers. They are both computers in exactly the same way. We havent gone to the length of a water-to-lcd display, but there's no reason we couldnt.
Water falling over a series of moveable rocks can (easily) be arranged to display the result of "1 + 1" on an LCD (eg., bucket/pixel, with water level = intensity, pipes to handle "graphics").
The use of the word wrong sounds wrong to me here. The wrongness should be about whether the model accurately, or just approximately, or not at all, shares with the modelled object that part of the behaviour we are interested in.
Semantics. Unless you disagree of course.
It also explains how we can keep adding complexity even if no new neurons are being created, since the branches themselves act like extra neurons.
> We model the brain as a multi-agent organization. Based on recent neuroscience evidence, we assume that different systems of the brain have different time-horizons and different access to information. Introducing asymmetric information as a restriction on optimal choices generates endogenous constraints in decision-making.
There's also Society of the Mind ( https://en.wikipedia.org/wiki/Society_of_Mind ) by Marvin Minsky (which is very readable)
> A core tenet of Minsky's philosophy is that "minds are what brains do". The society of mind theory views the human mind and any other naturally evolved cognitive systems as a vast society of individually simple processes known as agents. These processes are the fundamental thinking entities from which minds are built, and together produce the many abilities we attribute to minds. The great power in viewing a mind as a society of agents, as opposed to the consequence of some basic principle or some simple formal system, is that different agents can be based on different types of processes with different purposes, ways of representing knowledge, and methods for producing results.
[0] Just so we’re on the same page, when people talk about the brain as a small world network, the general idea is that a neuron in (say) auditory cortex mostly communicates with other auditory neurons, locally and in higher/lower auditory areas. It doesn’t talk to neurons representing (say) touch in the small of your back. Thus, there are far fewer than 86B! connections in the brain, but there are still an awful lot!
Specific patterns of concurrent input on a dendrite drive sub-threshold depolarization which is theorized to be key for sequence prediction.
I wouldn’t be shocked if consciousness were composed of hundreds or even thousands of NNs. Or even a tree thousands of levels deep.
A core tenet of Minsky's philosophy is that "minds are what brains do". The society of mind theory views the human mind and any other naturally evolved cognitive systems as a vast society of individually simple processes known as agents. These processes are the fundamental thinking entities from which minds are built, and together produce the many abilities we attribute to minds. The great power in viewing a mind as a society of agents, as opposed to the consequence of some basic principle or some simple formal system, is that different agents can be based on different types of processes with different purposes, ways of representing knowledge, and methods for producing results.
That can't be true otherwise it would take you a minute to form a conscious thought. Biological neurons are slow. If anything, artificial neural nets are 'deeper', going up to 1000 layers.
There are probably lots of huge differences between NNs and brains but this article is really making the case that the brain can be modeled as a big NN, just with a few thousand times more activations than neural cells.
This blog post via the Human-Centred AI research from The Stanford Institute, dealing with a similar subject matter, is wide-ranging, incisive and replete with sources.
https://hai.stanford.edu/news/intertwined-quest-understandin...
Is this functionally equivalent to having a two layer mini-network (that represents one brain neuron), with one neuron on top, and "child" neurons on bottom that mimic the grouping behavior? If this is true, then I would suspect our networks are already doing something like this automatically.
This should not be construed as denigrating the wonderful achievements of AI researchers. Just because what they do is inspired by the brain rather than isomorphic to the brain doesn't mean it isn't great work.
While we don’t have any proof of this conjecture (as far as I know) neither have we discovered any exceptions.
This also doesn’t rule out the possibility of non-physical or non-mechanical elements in the brain (dualism/vitalism) but frankly I don’t even entertain that notion.
Which is exactly my point — everyone is completely okay with those assumptions, without justifying that. I find it suspect.
How about showing physical processes are necessarily Turing computable, that is, justifying your underlying assumptions, before the straw man implication that I’m talking about dualism?
The mathematical equivalent of your argument is that because all finite-length approximations of a number are rational, the number itself must be rational — but this is untrue, in the general case. And in fact, for almost no numbers does a finite set of those rational approximations yield a general rule to predict the full structure of the number.
It’s therefore unclear that our limited scientific models being computable mean the underlying object they’re approximating is computable. But if we don’t know reality is computable, then we don’t know it can be simulated on a Turing machine.
Just assuming an answer doesn’t help us resolve the claim.
turing machines >= brain (since a brain is physical) and brain >= turing machine (by simulation argument)
The conclusion is the brain and Turing machine can do the same thing (brain = turing machine).
Another religious tenet with no observable basis.
Where did so many hackers get this misconception that computable and physically possible are proven to be the same? Many claim the Church-Turing thesis shows this. Have they never read it carefully?