Networks with selective attention already exist, but what if they can learn about themselves? Right now they cannot create any notion of self or "body" (defined as the boundary between the environment that can be predicted vs that which cannot), because their outputs have no causal effect on their inputs. There are no differences within the network that make a difference to itself, there is no intrinsic perspective.
Could this change, if, for example
- inputs are augmented with the network state (or derived version thereof)
- previous outputs of the network / external memory are fed back?
This seems to be the kind of self reference self awareness requires.
Also, do asynchronous networks have fundamental advantages over synchronous networks? What about static vs dynamic networks?