Brainwaves Encode the Grammar of Human Language
maxplanck.nautil.us
maxplanck.nautil.us
While we haven't observed grammar in non-human animals yet, whether they can learn and use it is very much still an open question.
"considering the rather limited number of experiments and the difficulty to design experiments that unequivocally demonstrate more complex rule learning, the question of what animals are able to do remains open." [1]
"when we consider how animal vocalizations are analyzed, the types of stimuli and tasks that are used in artificial grammar learning experiments, the limited number of species examined, and the groups to which these belong, I argue that the currently available evidence is insufficient to arrive at firm conclusions concerning the limitations of animal grammatical abilities." [2]
1: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3367684/
2: https://link.springer.com/article/10.3758/s13423-016-1091-9
When I took a Linguistics course at university, both the professor and text book were very emphatic that animals do not have (read: have never been observed using) language, they have communication. They went as far as to say that language was the defining characteristic of humans.
Comprehension of sentences by bottlenosed dolphins - https://www.sciencedirect.com/science/article/pii/0010027784...
First evidence that birds tweet using grammar - https://www.newscientist.com/article/dn20615-first-evidence-...
I think it's probably in how sophisticated it is. The bird example seems analogous to something like a handshake. A handshake has a sort of grammar to it, has to learned by observation/instruction, and if you scrambled it, people would be confused. However, I don't think the ceremony of handshaking qualifies as a language.
The dolphin example as well, while more sophisticated, isn't very when compared to what humans nearly universally do without effort. Though that could be a limitation of the study. Given enough time we may discover a dolphin language or teach them one comparable to what we can do.
Take what I said with a grain of salt though, I don't really know what I'm talking about here.
I take it all with a lot of salt until we have a wide range of animal languages down. One thing I found interesting is the recent work on dolphins has found entire images of things they have echolocated embedded in their clicks, suggesting a possible pictorial language.
Here's an excerpt from the summary -
>"We discovered transient wave patterns in the water cell that were strikingly similar in shape to the objects being echolocated. To further investigate the shapes in these Cyma Glyphs we converted the images to 3-D models. As far as we know, this the first time such a method has been implemented.
>Transient wave images were found for the cube, cross, flowerpot and a human by examining single still frames, or single frames acquired in bursts, usually where there was high power and dense click trains in the recording. The parameters involved in capturing echolocation images with the CymaScope include careful control of the acoustic energy entering the visualizing cell. Water requires a narrow acceptable “power window” of acoustic energy; too little energy results in no image formation and too much energy results in water being launched from the cell. As a result, many hours of work were involved in capturing the still images of the echolocated objects. The imagery for the human subject was captured in video mode and has been approximately located in time code. The image formed between 19.6 to 20 seconds into the recording and may derive from a set of dense clicks at approximately 20 seconds. The video was shot at 24 frames/second with a possible audio synchronization error of plus or minus 3 frames."
https://www.omicsonline.org/open-access/a-phenomenon-discove...
I wish the article fleshed out more details about the research. It looks like they found a correlation between the output of a specific ANN that seems similar to observed neural oscillations in the brain. The causality claim in the article seems premature.
Also curious if the neural encoding of grammar is consistent across different languages. So many questions about the specifics, but cool research for sure.
Initially my undergraduate CS advisor was excited by my idea, but only, it turned out, in the context of using it in genetic sequencing (because buzzwords). I gave him my honest opinion that adding my contrivance could only perform worse that existing genetic sequencing algorithms; my intent was to propose and investigate a way the brain might reasonably be recognizing language. He was entirely uninterested in this, and I got too busy with college / work and never developed the idea beyond a toy demo. Maybe I should look at it again...
The paper's conclusions:
"In sum, we remain relatively agnostic about the specific details of the required representational hierarchy because we do not yet know how to link the predicate calculus representations we use to natural language mental and cortical representations. What we are not agnostic about is the need for asynchronous time-based binding in order to produce oscillations in a neural network, as well as the need for representational hierarchy to produce the particular pattern of oscillations observed here and in"
The paper on evaluating if NLP parsers with representational hierarchies reflect what the cortical functionality:
"Our results naturally beg the converse question, as to whether any system with representational hierarchy could produce the oscillatory pattern of activation that [6] and DORA show. For example, could natural language processing (NLP) parsers, which feature representational hierarchy and were developed to specifically parse natural language in a machine, produce oscillations and the pattern seen in [6] and in our simulations? In principle, any system of hierarchy that is (de)compositional has the representational ingredients to encode units that could be fired in a sequence. However, we would argue that any given representational hierarchy could only produce oscillations if it were combined with time-based binding, which, as far as we know, no NLP system features. In DORA, time-based binding is the oscillation of activity throughout the network, which is part of the reason why the RNN did not show oscillatory activity nor the specific pattern from [6]. The representational structure of DORA (a (de)compositional role-filler binding predicate logic) is what makes the oscillations take the form that the data from [6] have. In terms of the specifics of the observed 1-2-4 Hz pattern, both [6] and our simulations are highly shaped by the word presentation rate of 250 ms/4 Hz. But without time-based binding, there is no mechanism to produce oscillations in a network, even in a NLP parser or other system with representational hierarchy."
Reminds of another proposed model from Gyuri Buszaki about how natural brain rhythms might be used to encode sentences.
https://arxiv.org/abs/1609.03971
> Efforts at understanding the computational processes in the brain have met with limited success, despite their importance and potential uses in building intelligent machines. We propose a simple new model which draws on recent findings in Neuroscience and the Applied Mathematics of interacting Dynamical Systems. The Feynman Machine is a Universal Computer for Dynamical Systems, analogous to the Turing Machine for symbolic computing, but with several important differences. We demonstrate that networks and hierarchies of simple interacting Dynamical Systems, each adaptively learning to forecast its evolution, are capable of automatically building sensorimotor models of the external and internal world. We identify such networks in mammalian neocortex, and show how existing theories of cortical computation combine with our model to explain the power and flexibility of mammalian intelligence. These findings lead directly to new architectures for machine intelligence. A suite of software implementations has been built based on these principles, and applied to a number of spatiotemporal learning tasks.
YMMV
> Our artificial Feynman Machines have several interesting properties which distinguish them from existing Deep Learning and similar systems. In particular, due to the much higher density and locality of processing, a Feynman Machine-based system can perform at least comparably while dramatically reducing the footprint in computational power, training data and fine-tuning. Feynman Machines can be arbitrarily split into modules distributed across clusters and the Internet, and systems running on low power devices such as phones can be dynamically and robustly augmented using low-bandwidth connections to larger networks running in the cloud. Models can be trained on powerful infrastructure and transferred for operation and further custom learning on smaller devices. Importantly, the same architecture can be used as a component in unsupervised, semi-supervised, fully supervised and reinforcement learning contexts. A variant - the Routed Predictive Hierarchy - is described, which allows a Feynman Machine to directly control a traditional Deep Learning network by switching it to use spatiotemporally-selected subnetworks.
This is the abstract of the previous paper that detailed the theoretical basis [0]:
> Reverse engineering the brain is proving difficult, perhaps impossible. While many believe that this is just a matter of time and effort, a different approach might help. Here, we describe a very simple idea which explains the power of the brain as well as its structure, exploiting complex dynamics rather than abstracting it away. Just as a Turing Machine is a Universal Digital Computer operating in a world of symbols, we propose that the brain is a Universal Dynamical Systems Modeller, evolved bottom-up (itself using nested networks of interconnected, self-organised dynamical systems) to prosper in a world of dynamical systems. Recent progress in Applied Mathematics has produced startling evidence of what happens when abstract Dynamical Systems interact. Key latent information describing system A can be extracted by system B from very simple signals, and signals can be used by one system to control and manipulate others. Using these facts, we show how a region of the neocortex uses its dynamics to intrinsically "compute" about the external and internal world. Building on an existing "static" model of cortical computation (Hawkins' Hierarchical Temporal Memory - HTM), we describe how a region of neocortex can be viewed as a network of components which together form a Dynamical Systems modelling module, connected via sensory and motor pathways to the external world, and forming part of a larger dynamical network in the brain. Empirical modelling and simulations of Dynamical HTM are possible with simple extensions and combinations of currently existing open source software. We list a number of relevant projects.
Edit:
Apparently, the project is called Ogma and they have a youtube channel. Here's their video [1] of a toy car learning to self-drive after a few laps of a person driving it with an xbox controller. It runs on a NanoPi Duo. Not sure if that's noteworthy or not.
What about, for example, differences in languages, culture, personal background, etc.