There must be something "turing complete-ish" about neural nets. As in although the primitive element of a human neuron is complex, you can achieve identical computational power with a much simpler model.
The problem is that it can require an exponential amount of time as well as data to actually get a decent network--and, of course, there's a high risk of it being overfit to idiosyncracies in the data. The standard approach to neural network training is to convert it into a gigantic calculus problem and brute-force it, and this doesn't always result in timely convergence or even a generalizable model.
I'm thinking the current primitive of the neuron in ML may be a bit flawed as a foundational primitive but I certainly don't think we need to model the complexities of a biological neuron.
We have a math function that defines atoms. Atoms can be simulated. If all things are made of atoms then all things can be simulated. If all things can be simulated then YES all things can be defined by a mathematical function including the brain.
In fact, it's been done already. Researchers have already simulated part of a mouse brain. The reason why they can't do humans is only scale. Humans have far more neurons so it's harder to do a brute force copy and the computing budget is more larger. See the Blue brain project, it's actually old news.
If there's something more, we don't have the tools to prove it.
This is actually not true and can logically be shown. We can model atoms with computation. Thus if all matter including the brain is made up of atoms then it can be simulated via a math equation.
The idea that we understand what physical matter really is, how it works, whether it can produce qualia, et al, is simply off the mark. Of course most physical systems can be reasoned about macroscopically and are relatively deterministic--we can understand how computers work because the functionality is almost entirely classical--and we know that quantum mechanics is both mathematically and experimentally valid, but we not at all gotten to the bottom of this.
In fact, one can argue that physics as we understand it can't be used to explain qualia. Physics requires symmetries (translational, rotational, spacetime) and asserts that there is no "privileged" reference frame that defines, say, the "objective" origin or x-, y-, z-axes. If you have two pure kilogram spheres of iron--of course, nothing macroscopic is actually 100.00% pure in practice, but it is physically conceivable and possible--they are exactly the same. One is a perfect substitute for (or copy of) the other. Consciousness likely violates this. We believe (although, without proof) that a perfect physical copy of a person would harbor a separate consciousness and that they would in fact diverge. This suggests that consciousness is a product not of the physical state--structural equality--but is driven by something more like pointer equality. The question is what it's "pointing" from. And, of course, we have no idea; we can't even prove that this is the case.
No we don't. The probability wave can be simulated. Decoherence cannot be simulated unless we have a true random number generator. But this is a simulation. We can perfectly simulate, but not perfectly recreate. In fact even with entropy for random number generation derived directly from quantum foam you cannot deterministically recreate the behavior of any atom. You can only "simulate." because the nature of the math equation involves randomness and perfectly imitating randomness involves luck.
>The question is what it's "pointing" from. And, of course, we have no idea; we can't even prove that this is the case.
Your statement is also highly unlikely. It's like saying the universe was created by a turtle. You can't prove or disprove it. But it can still be considered absurd.
>One is a perfect substitute for (or copy of) the other. Consciousness likely violates this. We believe (although, without proof) that a perfect physical copy of a person would harbor a separate consciousness and that they would in fact diverge.
The word consciousness is loaded, poorly defined and complicated. It is a language phenomenon that tricks people into thinking consciousness is some profound concept. It is not. It is simply a series of thousands of attributes related to intelligence such that if you have all the attributes you are conscious if you are missing one attribute, you are not conscious and nobody fully agrees on what all these attributes are. That's it. So they argue about it thinking it's some philosophical debate when they're really just arguing about vocabulary and trying to map out the definition of a loaded word.
It's a big trick.
A copy of consciousness would be like a copy of a computer. There's nothing magical about it. Would the copy of a computer behave exactly identical to the original? Maybe not "exactly" but they would be similar... but there is nothing profound about that.
You're getting hung up on the pointer. So if I copy a computer will the "pointer" suddenly be confused? No. It won't be because there is no "pointer" to a computer just like consciousness is NOT a pointer. It's actually shows bias the fact that you have this notion of a pointer. Think about it. If I copy a rock what happens to the pointer of the rock? When I copy a printout what happens to the pointer to that printout? Why does consciousness arbitrarily out of nowhere suddenly have a possibility of a pointer? What is it that makes you think this unprovable concept of a pointer applies just to consciousness but you don't really think about it for other mundane forms copying?
It's bias for the word "consciousness" likely caused by the "big trick" I talked about above.
You are a physical being you are not a pointer pointing to a physical being... the brain and your soul are one in the same thing. This makes sense because this is how we assume all OTHER things in the universe work. There's no pointer pointing to them so why is there suddenly a pointer to consciousness? You said it's unprovable, but to even entertain this possibility is like entertaining the possibility the universe was created by a turtle. Possible... but unprovable.
We aren't fully clear about what this model of intelligence is, but I'm instinctively sure that it won't have to involve anything as complex as biological neurons that communicate through thousands of different pathways and electromagnetic side effects too.
That's just a feeling you have.
1. We know how natural selection works, and it clearly often produces unnecessary parts for many things.
2. There is physical evidence of evolution creating unnecessary and inefficient parts and components.
Additionally the selection criteria is simply survival. Not efficiency. As long as the "processing modalities" in the brain aid in survival the selection process does not interfere. Thus all kinds of arbitrary processing modalities both efficient and inefficient can occur so long as the modalities do not detriment survival. Why does stupidity evolve? Well stupid people aren't smart enough to create nuclear bombs to kill themselves. Thus stupidity is a possible trait, similar to how inefficient and unnecessary "processing modalities" are possible, mayhap being stupider aided in survival or didn't contribute anything at all. Just looking at people responding to me on HN, alot of them are pretty stupid (physical evidence). Not pointing out which one, certainly not you.
2. There is (alot) of physical evidence of the above. For example, the wheel is a very efficient way of traveling yet no animal has evolved some form of a wheel. The reasoning is actually more complicated about why a wheel hasn't evolved but the general idea I described above still applies... both natural selection and random mutation were not able to generate the series of intermediate traits required to form wheels.
Thus from the reasoning above it is very possible for the brain to evolve with unnecessary and inefficient "processing modalities".
Yes this idea did begin as a gut feeling. Subconscious processing is often times correct and logical. It is unwise to constantly mistrust it. It is always wise to analyze these "feelings" in attempt to break down the logic behind why it occurred.
What my self analysis tells me is this: Given the fact that MANY of our mathematical models for physical processes involve elegant and straightforward primitives it follows that intelligence will also VERY likely be such a model as well and very likely much more simplified then the human brain.
For example, look at cellular reproduction. Just imagine if your phone could produce a smaller phone, which would develop into a fully functional phone. And imagine if it could do that using energy which was converted from breaking down fruits and vegetables. These are things that every living thing can do, but it's way beyond the scope of what our technology can accomplish.
There are many such miracles of biology. For instance, a gecko's feet act as an adhesive in a way that artificial imitations have not been able to match.
Wheels are also great unless they aren't. If we had evolved in an environment where we had lots of smooth roads to roll around on, maybe some animals would have evolved wheels, but instead we evolved legs which are much better for getting around uneven and unpredictable terrain.
> Thus from the reasoning above it is very possible for the brain to evolve with unnecessary and inefficient "processing modalities".
It's possible, but what evidence do you actually have that this is the case? How do you know the brain hasn't been subject to selection pressures forcing it to have evolved in an extremely efficient fashion?
> Subconscious processing is often times correct and logical.
Is it? I thought you said your brain was just an inefficient accidental mess? Why would you trust it? Or is your brain maybe incredibly powerful and capable, enough to create abstract and complicated emergent phenomena like intuition, which are so far impossible to understand analytically, or indeed to reproduce artificially?
> What my self analysis tells me is this: Given the fact that MANY of our mathematical models for physical processes involve elegant and straightforward primitives it follows that intelligence will also VERY likely be such a model as well and very likely much more simplified then the human brain.
This is a massive leap of faith and I don't know how you can possibly argue otherwise!
YOu need to explain how it's a leap of faith. That means low probability. So you need to explain the sample space and the numerator.
For me I explained. The sample space is ALL models for physical processes. Given that most of these models are elegant with efficient and simple primitives. It follows via probability that modeling intelligence by probability will most likely be the same way. See what I did there? I actually logically explained why it's not a leap of faith.
>Is it? I thought you said your brain was just an inefficient accidental mess? Why would you trust it?
First off being inefficient is different from being correct. The brain can be correct AND inefficient at the same time. It can also be correct about some things and incorrect about other things. This is obvious if you look at the world around you. People are often wrong. They are also often right. Evidence is everywhere and obvious.
> But biology has also produced systems which are far beyond what we've been able to engineer.
Sure it does. But I talk in terms of probabilities and possibilities. I am addressing the "leap of faith." I am saying because such and such is extremely possible... my statement is not a leap of faith. In this case I am saying inefficiencies are very possible. Thus my statement is highly realistic.
I never made a statement about anything absolute. I never said that absolutely the brain is inefficient. I simply said my statement is categorically not a leap of faith because it is VERY possible. And i provided logical evidence supporting this.
1. Biology can produce things which are incredibly smart and efficient. It can also produce things which are dumb and inefficient.
2. Human intellect and consciousness is one of the most complex nuanced phenomena we are aware of in the universe.
3. The human brain is one of the most complex things we know about in the universe.
4. We know very many things about how the brain processes information. We know structure and function are heavily intertwined. We know that the brain uses virtually every aspect of neural biology, from macro structures down to intracellular mechanisms, to facilitate information processing.
So we know the biology of the brain is incredibly complex. We know the output of the brain is incredibly complex. We know of many instances where the complexity of the biology contributes meaningfully to how the brain processes information. That is all evidence to support the belief that the complexity of the brain is meaningful with respect to the production of human intelligence.
It seems that you want to believe that most of that complexity is not meaningful, and your only evidence is "you had a feeling" and "biology is inefficient sometimes".
If you want to prove it's not a leap of faith, you have to present examples in which the information processing of the brain is actually inefficient, or how it can be replaced by a much simpler system. Otherwise it's all just a guess.
By your second sentence it follows that such structures aren't a leap of faith. It is very possible.
2. Human intellect and consciousness is one of the most complex nuanced phenomena we are aware of in the universe.
complexity isn't indicative of efficiency.
3. The human brain is one of the most complex things we know about in the universe.
Same as above.
4. We know very many things about how the brain processes information. We know structure and function are heavily intertwined. We know that the brain uses virtually every aspect of neural biology, from macro structures down to intracellular mechanisms, to facilitate information processing.
But we don't know whether the entire structure is efficient.
>That is all evidence to support the belief that the complexity of the brain is meaningful with respect to the production of human intelligence.
But it is not evidence supporting the thesis that the brain being inefficient is a leap of faith.
>It seems that you want to believe that most of that complexity is not meaningful, and your only evidence is "you had a feeling" and "biology is inefficient sometimes".
The feeling has been rationalized. I explained the logic, it is now backed up. The criterion here is not absolutist. The criterion is just stating whether or not such a scenario is likely. IS it a leap of faith or is it not? I've sufficiently proved that the brain being inefficient is clearly NOT a leap of faith.
>If you want to prove it's not a leap of faith, you have to present examples in which the information processing of the brain is actually inefficient, or how it can be replaced by a much simpler system. Otherwise it's all just a guess.
Easy we have many ML models that already beat humans at certain tasks.. Those models use much less neurons then the entire brain itself and follows a much simpler model. See spam filters. It shows that a simpler model is already possible for certain aspects of intelligence.
I disagree with you. I believe I've shown enough evidence to move the probability out of "leap of faith" territory. I just threw in the example up above because you requested it.
> There must be something "turing complete-ish" about neural nets. As in although the primitive element of a human neuron is complex, you can achieve identical computational power with a much simpler model.
So the assertion is, that you can achieve identical computational power to a neuron with a much simpler model.
That is a leap of faith. There is insufficient evidence to conclude this. If you said "I believe there might be" or "it's possible that" I would not have taken issue with it, but there is a very high burden of evidence to assert this belief. Without that evidence you are operating on faith.
Until we can successfully reproduce something approximating human intelligence, or until we can postulate a credible mechanism by which human intelligence arises, it's a very very big leap to assume we understand anything about what is or isn't required to produce it.
> complexity isn't indicative of efficiency ... But we don't know whether the entire structure is efficient.
We don't know the complexity is efficient, and we don't know whether it's required. What we do know for a fact is that a human brain is capable of producing human intelligence. We don't have evidence that any other system is in fact able to produce human-like intelligence.
> But it is not evidence supporting the thesis that the brain being inefficient is a leap of faith.
I don't have to produce evidence that it's a leap of faith. You made a claim that is not supported by evidence. You have to provide evidence for your claim, or else it is defacto a leap of faith.
> The feeling has been rationalized. I explained the logic, it is now backed up. The criterion here is not absolutist. The criterion is just stating whether or not such a scenario is likely. IS it a leap of faith or is it not? I've sufficiently proved that the brain being inefficient is clearly NOT a leap of faith.
Yeah but you can rationalize just about anything. I could provide a rational argument for why God exists. It's still a leap of faith to believe in God.
You have a rationalization, but it falls far short of being any kind of rigorous proof.
> Easy we have many ML models that already beat humans at certain tasks.. Those models use much less neurons then the entire brain itself and follows a much simpler model. See spam filters. It shows that a simpler model is already possible for certain aspects of intelligence.
All ML models prove is that we have been capable of producing machines that are capable of pattern recognition. It's a massive leap of faith to generalize this result to assert the belief that neurons in general can be modeled at a much lower level of complexity. It may be the case, but there is insufficient evidence to conclude that. It's just your belief based on gut instinct, and the fact that you've talked yourself into it.
I use the word "Must" as in "highly likely" if my words didn't convey the probability then my bad, but I assure you now that my intent is conveying a probability.
> There is insufficient evidence to conclude this.
There is enough evidence to show it is sufficiently high probability for consideration.
>If you said "I believe there might be" or "it's possible that" I would not have taken issue with it,
I did say this MULTIPLE times in subsequent responses. I am telling you now that is 100% what I meant. Now that we're clear, we are in agreement. This conversation is over.
Agree to disagree.
It is still proof by contradiction, what you say is categorically not true.