Hacker News Reader

One of the most important thing to remember is the efficiency of the human brain is layered. The ancient parts (vision for example) have been honed to an amazing level of efficiency so even the most powerful computers can't come close, while the other modules (say simple arithmetic) are so inefficient that a $2 calculator needs to uses only 1/100,000th the energy to solve the same problem. The general rule is anything we find easy as humans to accomplish is amazingly hard and ancient, while anything we find hard is relatively easy and recent (evolutionary).

One thing to point out that even if the "module" that does arithmetic is slow, the brain understands the context of that calculation.

2 + 2 can be viewed as a computation that happens in nature and our brain all the time.

But the real magic doesn't lie in the pure computation. Our brain understands what that 2 means in the context of all the knowledge of the entire Universe. 2 cows, 2 sheeps, 2 planets ?

Understanding context is what we are really good at, as said in the article "prior" knowledge. So its not surprising that a cheap calculator can outsmart me in computation when it doesn't actually compute the context of its computation, which is sort if cheating.

I was trying to stay away from "meaning" since it gets messy, but if you just compare the energy used by sub-conscious human arithmetic with that expended by a simple cpu then the cpu is massively more efficient. We find mathematics far more mentally taxing than visual observation purely because we are so bad at mathematics, not because visual processing is easy.

Arithmetic savants and trained arithmeticians can come a lot closer to the calculator, showing that the usual human brain layout and/or architecture is a really poor fit for performing this type of calculations.

There's also some evidence that the "efficiency" of the more powerful parts of animal brains (including humans) simply comes from a sort of n-dimensional conditional railroading, which would work like this if my understanding is correct: if you map (B/W) visual inputs to XY axes of a slice of neurons, then when a line is formed, this fires up a line of neurons, which allows a direct connection between the Z±1 neurons of the first and last neurons on the mapped line, so this connection becomes the line. These Z±1 neurons have pathways to various other nearby other potential Z±1 neurons; a connection lighting up between Z±1 neurons that don't have a straightline connection over the XY map could detect different shapes. Then the Z±1 layers and their connections could be creating highway maps between both Z±2 neurons and diagonally-offset neurons that don't "fit in the grid" we're imagining here (remember, we're in 3D and neurons don't form a homogeneous grid, so this is all abstraction anyway).

These second-level connections, arguably identifying something like shapes, have connections in such a way that the next level up detects something else, etc. (insert magic we don't understand), until the detection results in recognizing a specific object, while the side-channel connections we haven't mentioned yet but kept occurring at every level connect to a different side-channel ZY map for, say, the position of the object (with its own layers of processing that eventually connect to the object concept neuron so that the "position" is associated with the recognized object), and/or perhaps other side-channel maps or parallel XY mappings for various other things. And of course, a lot of these things connect to completely different parts that we ignore here, and a lot of the information indirectly makes its way to the conscious mind in this way; the brain doesn't compile a complete vision report and submit it to the conscious mind in one download, it's all using the same hardware.

In essence, in order to figure out the brain, we would need to figure out how a single neuron actually works.

I think that will take us 500 years, give or take a few.

Why do you think that would be the case?

You could say the same about every tool that we use, which we don't totally understand. It just needs to work.

No, we wouldn't need this. How we could come to figure out and understand "the brain"[1] without understanding its smaller components is worth several science classes mostly involving maths, most of which I haven't mastered and thus won't arrogantly attempt to explain. We've managed to figure out other complex systems without understanding their components in detail. For one point of comparison, we'd already had a pretty good idea of how inertia and motion worked (Newton's laws) long before we understood in detail the atoms and forces acting upon/between them.

Then there's also the problem of your estimate. 500 years is a rather exaggerated timeframe. Twenty years ago I could've had the most prominent field experts tell me humans would never in the next three hundred years figure out biology even for the simplest of living organisms, because they were simply too irreducibly complex for that. And then a few years ago we simulated an entire worm's nervous system and gave it enough of a body for it to use, move around, and receive input from its entirely-fictional environment. Now we've got people working on doing the same thing with a cat brain. We're nearing breakthroughs in creating fully-synthetic, fully-functional animal organs that you can essentially real-life drag'n'drop to replace a failing natural organ without complications.

Knowing the above, are you really sure it's not 15 rather than 500 years?

[1]. (which is about as meaningful as saying we understand "weather", which is really a huge messy amalgam of many completely unrelated things ranging from fluidic motion and thermodynamics all the way to plate tectonics and even anthropology, after a fashion)

I can pick up a rock and throw it at the water so it skips across the surface. It takes less than a second. How many calculations did I just do?

We are 'slow' only when we formalize the problem.

> How many calculations did I just do?

Probably not that many. Let's say you create a water-skipping robot with two different AI programs in it and a shaky, not very precise arm (but still as good as a human arm in terms of specs).

One is custom-made to the physics of the problem, and will calculate the angles and the forces and the pressures and the candy tasty physekz all over, and then decide on a particular motion of the robot arm, and the rock will skip. It'll take a lot of computing power, but it'll work.

The other just has a goal, to see the rock skip, and tries things at semi-random (though it has the general knowledge that it can be done and that it has to throw the rock towards the water in a particular way for it to happen and that it can control the outcome), and tries to figure out patterns between what it did and what happened as a result. Eventually, it has a particular tactic, a certain set of instructions, which could be "down down down the left thing and up up right down down the right thing and the up thing does down down up push push down push twist-force-2 at the same time", an entirely not at all complicated set of instructions with very little computation. This signal is sent through other nodes that might not have the perfect signal number, and eventually this outputs to the arm's motors... which nevertheless manage to make the rock skip, because the motion here is almost the same as the motion of the first AI, yet this one was obtained by eliminating the ones that didn't result in the rock skipping.

TL;DR: You don't do that many calculations on the spot. The "maths" in most situations like this is done by elimination throughout all the attempts you've made in your life to control a throw. Subsequent throws once you're already a practiced rock-skipper just involve firing "the same neurons as usual" which send "the same signals as usual" to your muscles, which result in the same skipping as usual, with very little math. If you already know that 2x^5 + 20 = 110 implies x = 45 from doing the same calculation twenty times in the past hour, your brain isn't performing "calculations" anymore, it's just repeating a pattern that's already there in your brain like a table lookup.

I tend to disagree with the idea that we don't have the computational power equivalent to the brain. The reasoning is that, yes, all the things you say are true about having billions of slow neurons, but that makes the assumption that their only fault is that they are slow. I don't necessarily agree with that assumption. The evolutionary process doesn't find true maxima, it finds local maxima. The key is that the neuron is good enough for the task of intelligence and planning and pattern detection. However, the architecture is most likely highly inefficient. The reason why they are so good comparatively is because we just kept adding neurons, either by increasing the size of the frontal cortex and by the folding, so as to jam more in there.

That is not an architectural design per se, and I highly doubt that it is optimal. Rather, it is good enough. Now, the Von Neumann computer architecture is of course lacking. Parallel computing as we know it works, but it's a total nightmare, and hardly competes with the incredible parallelism in nature. But I do think that there is sufficient power now to really feasibly compete with the brain, intelligence wise.

All this with a grain of salt. I have been reading stuff in the weird corners of AI research. I think this article is fundamentally a straw man. (I believe that...) Deep learning and probabilistic machine learning are fundamentally flawed for strong AI. Jeff Hawkins is a well respected AI researcher that I think seems to agree. Another problem is of course that these are all trained in a supervised fashion. Google does well because they have huge tagged data sets. However, the brain doesn't work by training on tagged data sets, it can learn on it's own, unsupervised. So long as we keep pointing mainstream AI down the supervised statistical machine learning path, we will always be far away from strong AI.

I don't agree that brains can learn on their own, unsupervised.

Firstly, the brain has a basic feedback mechanism, which can be thought of as basic supervision: it receives quick feedback on certain kinds of hardware issues its actions caused. Most computers don't even get a similar level of feedback (eg, that they tied up the cluster on junk computations).

Secondarily, modern humans are booted through this mechanism by already running humans, in what is direct supervised learning: from the time we're infants -- and for the next two decades -- our actions are supervised, commented on, corrected and our exposure to materials is metered and selected to create a (theoretically) optimum consumption plan.

That we get any results out of computers with a couple months of training when comparing them to humans with a couple decades of training is testament to the fact that computer learning is orders of magnitude more effective than human learning.

I find it very strange that people leave out the 2 decades of hardware tuning and supervised knowledge building that humans get when discussing about how awful it is we having to train our machines if we want them to be smart.

Brains can definitely learn without supervision. Observe any child playing; they explore and learn from stimulus in ways far more sophisticated than our current machine learning models.

An interesting aspect of brains vs machines is how brains can learn from other brains indirectly. When someone tags a database for a machine to learn from, that is a form of direct communication (the kind machines are good at). A crow can watch another crow use a stick as a tool, and in turn learns that a stick can be used as a tool. This requires a complex understanding of the situation.

If you feed one AI a tagged data set to train it, I think you would be able to easily hook it up to an untrained AI to train the second AI. As in, pass the first AI an un-tagged input, get it's result, and pass the second AI the same input tagged with the first AI's result. That would be essentially the same thing as a crow learning from watching another crow, aside from the fact that the crows are self-directed.

> I don't agree that brains can learn on their own, unsupervised.

Facts don't require your agreement. They're still facts.

You're suggesting that the "2 decades of hardware tuning" is equivalent to programming, and that just isn't the case. While parents definitely provide instruction, each child's brain is fully on its own to program itself. It teaches itself how to make sense of the visual signals it's receiving from the retina... no one teaches a child up from down, how shading and colors differ, how to use both eyes in tandem to provide depth perception. It's all automatic and 'magical' in a way, because literally NOTHING you can do as a parent can speed that process up in any meaningful way. The same is true for hearing. You can't just upload your current understanding of the world... the closest you can come is by trying to help them along mechanically, by holding them up as they 'walk' their legs, for instance, but their brain is still figuring it out entirely independently. Showing the child how to move their legs is NOT the equivalent of showing them how to detect orientation using their inner ear, or teaching them how to send impulses to their muscles in the right order and at the right time, or teaching them to predict momentum so that they don't just fall on their face.

> computer learning is orders of magnitude more effective than human learning

The only reason one could even ATTEMPT to make that claim is that a computer's learning is something that can be copied digitally. A single machine can't even approach the learning abilities of the brain. Computers are so fundamentally different that it's not even a fair comparison to the computer... the brain will win every time.

> I find it very strange that people leave out the 2 decades of hardware tuning and supervised knowledge building that humans get

It's because it just isn't relevant. Take the most advanced AI available today, slap it in a robot and spend 2 decades raising it. I guarantee it won't end up anywhere close to a human in abilities or intelligence. If you can't see that, you're being intentionally obtuse.

The last part was funny.

Modern robotics has been around for more than 20 years and there is no autonomy whatsoever.

It might surprise many people who claim that AIG is 20-30 away from now that in robotics, something as simple as cup stacking is a major problem. If you can figure out how to do that you will get slightly famous.

The brain and a computer work so fundamentally differently that its impossible to provide a single framework to compare them. This is why I used the metric of computational complexity but even that is not at all accurate.

I think due to the tremendous amount of automation we have seen in the past 10-20 years, many programmer have difficulty appreciating the awesomeness of biological intelligence. Rather than view the success of automation as a failure of how human education works.

By fully appreciating how complexity of the biological intelligence can we start to learn from it and append it.

Totally agree! I think we have deviated too far from biological models in order to achieve marketable results. Google is satisfied with probabilistic machine learning because it will still make them plenty of money being able to tag images. This is why I admire the approach Jeff Hawkins is taking by going back to the biological model for more inspiration.

Of course, speaking from physics - yes. Its a local maxima.

Its one of the shortcoming of natural selection and genetic programming.

We have better tools to do sequential computation than biology can match. And soon we will be able to harness the power of quantum mechanics to do our bidding.

But the point I am trying to make is whatever approach we use, sequential or parallel, Just in terms of computational complexity. The brain's capacity is greater than the sum of the computers humans have ever produced. Just using that metric we are still far away from having the tools to simulate brains, how long will it take for Intel to compress the complexity of ALL computers ever produced into a single chip ?

Surely its not tomorrow ? maybe 20-30 years ? I do not know.

What I do know is in 2015 it seems that there is a lot of work that needs to be done just from a hardware perspective.

I would agree that the computational complexity of every single quark in every single atomic particle in every atom in every synapse is far far greater than the computational power we poses at this day. I just think it's a red herring to chase down that level of computational completixy (for now. True physical simulations truly amaze me, and I can't wait to see them be more granular and accurate). When it comes to intelligence, I think we are mostly missing the algorithm, not the computational power. I think once somebody finds the algorithm (which may be very soon in my own opinion), we will have more than enough computational power to blow the mind away by orders of magnitude.

There's a lot of structure in how the brain is wired that is is baked in when you're born. The visual cortex is in the back, the hippocampus is in the middle, the frontal cortex is in the front. You do have unsupervised learning, but you start out with highly tuned hardware. Is that not some form of basic programming?

yeah, absolutely. But that's the power of millenia of evolutionary tuning, which we have to intelligence to design ourselves, removing the inefficiencies. That then drops the computational power required for equivalent behavior.

I agree that the current Von Neumann architecture is probably not adequate. However, I think there is some active research in this area.

Here are a couple links:

From today actually. Memristors are an exciting development if they can scale production: http://www.technologyreview.com/news/537211/a-better-way-to-...

A startup working on this: http://brainchipinc.com/

IBM's TrueNorth Architecture: http://www.research.ibm.com/articles/brain-chip.shtml

All these development are really excellent.

The more people who are interested in solving these problems the better.

Neural Networks are awesome - but modern AI tells us that there are also various other techniques, The problem with Neural Networks is they are essentially a black box and many mathematicians and engineers are not happy working with black boxes.

Right now the models for synapses and Neurons are very simple compared to what actually is happening in them. But regardless the approach is not in vain.

(My current research involves autonomous motor control with tactile feedback, which is very far away from AGI)

Can you tell me more about your research? I became really fascinated with motor control after reading an introductory neuroscience book a while back. I feel like this is not an area that receives as much attention as it should.

True. However we know shockingly little of how neurons actually do computation. Everything a neuron does it does so with a combination of electrical and chemical signals. This is why it's slow, but it also introduces considerable noise in the process.

Also, because they're usually in some skull-like object, and because neurons are incredibly hard to culture, we simply lack the tools to observe them in their natural environment to the resolution you might want. Simple questions like how often a neural signal fails to cross a synapse in a brain are remarkable difficult to answer. Do we need to know how every neuron is connected to every other neuron in the brain to understand it? Even if we had such a map, would it be missing something so crucial as to be useless (for example how glia interact with the neuron)? Nobody can say for sure right now.

The human brain may indeed be beyond all computers.

On the other hand, the brain and eyes of the fruit fly do amazing things. I think it's reasonable to say that vision is lacking both hardware and algorithms at this point.