How 'deep' are the networks used in something like DeepMask, and how does it compare with the number of layers of the human brain?
Layers are an abstraction we use to make artificial neural networks dramatically faster to work with using linear algebra. By keeping each node interacting only with the layers above and below, we make the computations a lot nicer for our model of computation.
In the brain, neurons link freely to other arbitrary neurons based on adaption processes we (or at least I) don't really understand.
The brain also lacks a clear idea of direction to count the layers along, since it has innumerable different inputs coming in at all times, and the resulting signals interact all over the place.
The most meaningful analog would probably be to ask "How many neuron firings typically occur between an external stimulus and a response to that stimulus?" Even that is extremely rough though, because through evolution a lot of 'short circuit' structures have formed in our bodies. The gag reflex is obviously triggered by sensory input, but it probably doesn't check with your frontal lobe before firing the appropriate muscles.
A real neuron takes in the order of 10ms to integrate and fire to the next neuron. Many subconscious reactions take less than 1 sec, which leaves time to a chain of length less than 100. Note that those neurons are not strictly arranged in layers.
The human visual cortex has 10^12 synapses [1]. One popular 2015 deep learning net (ResNet 152-layers) used 10^12 FLOPs to classify objects in one image (but less weights.)
In terms of depth, we're there. In terms of breadth, it will take several years. But the brain does things very differently. For example, it has top-down signals during "prediction."
A Noob question:
If it reaches the "breadth" of human brain, how close will that to the "Skynet becomes self-aware" moment.
What would it tell us about human, our society after it study, analyze millions, billions hours of FB, youtube videos?
> To capture general object shape, you have to have a high-level understanding of what you are looking at (DeepMask), but to accurately place the boundaries you need to look back at lower-level features all the way down to the pixels (SharpMask).
It's several orders of magnitude more "neurons" and "connections" than even the largest ANN's