Brains and algorithms partially converge in natural language processing | Hacker News Reader

Brains and algorithms partially converge in natural language processing

nature.com

79 pointsAnon8410 comments

As a computational neuroscientist who has used ANNs to model brain data for 20 years, I want to suggest another way to think about these sorts of brain-ML/ANN comparisons: successful language comprehension requires representing the underlying statistical structure of language. Humans who learn to comprehend language must learn about the underlying statistical structure of language. ANNs that learn to comprehend language must learn the underlying statistical structure of language. Comparing human data to ANNs only really makes sense in the context of that statistical structure. What aspects of this statistical structure are learned by both machines (i.e., brains and ANNs)? What aspects of this structure are learned differently by the two machines? When the two machines learn different aspects of statistical structure, is this due to differences in architecture, training, or something else? A good understanding of the stimulus statistics is essential for interpretation.

> Humans who learn to comprehend language must learn about the underlying statistical structure of language

This comes up a lot, but is the evidence really there for it? Granted, we (humans) have mental models for "this is most likely what this person means", but I don't think that says a lot about the "underlying statistical structure of language". It seems to me like by the time such statistical probing takes place, for instance trying to decipher what someone means, the language learning already took place. Or do you mean something else?

Note: I am not an expert in this field. I studied some papers back in college addressing these topics and the evidence back then (out of the generative grammar school) was fairly convincing in refuting this statistical learning assertion. I wonder what papers or research you had in mind here.

Peter Norvig and Noam Chomsky ~famously clashed on this, and Norvig's argument shared a lot of similarities. https://norvig.com/chomsky.html

> we analyze the brain activity of 102 healthy adults, recorded with both fMRI and source-localized magneto-encephalography (MEG). During these two 1 h-long sessions the subjects read isolated Dutch sentences composed of 9–15 words. After quantifying the signal-to-noise ratio of the brain responses, we train a variety of deep learning algorithms, extract their responses to the very same sentences and compare their ability to linearly map onto the fMRI and MEG brain recordings. Finally, we assess how the training, the architecture, and the word-prediction performance independently explains the brain-similarity of these algorithms and localize this convergence in both space and time.

> We find that (1) a variety of deep learning algorithms linearly map onto the brain areas associated with reading, (2) the best brain-mapping are obtained from the middle layers of deep language models and, critically, we show that (3) whether an algorithm maps onto the brain primarily depends on its ability to predict words context

This is really cool research! Imagine wearing a VR like headset that records fMRI/MEG signals and then instantly transcribes the words you’re thinking of into your text editor. Neuralink may have some competition eventually if these findings generalize and can be used in a wearable fashion?

On another note, I believe masked language modeling was originally proposed in the BERT paper (Devlin et al 2019), as a way to learn contextual word embeddings using a transformer architecture (ie autoencoder with denoising reconstruction objective).

This paper seems to suggest the brain may behave somewhat like a denoising autoencoder. If these correlations can be leveraged, the implications could be staggering.

Would be really interesting to see how well it works for various people.

Eg i feel i have a very.. noisy mind. Maybe some ADHD, but i don't like to hop on diagnosis trains. Point is, unless i am uniquely "in the zone", i often feel my productivity is in bursts between thoughts wandering mostly randomly.

If i had the ability to just transcribe everything i think i suspect it would need a huge filter to actually get useful output.

Likely related, i often have to stop and re-order my thoughts during conversation or writing. I will jump through a sentence to "the point", and then have to back track and actually add the grammar/words necessary to actually convey "the point". It seems my thought process doesn't like to dilly-dally on the communication. It's very binary if you will. I either have the solution/point/etc, or i have a blank space where i'm trying to think of the solution/point/etc. Once i have it, then i have to back track and re-order everything to actually be able to convey the thought.

This thought-ordering issue becomes much more complex if the problem is multi-tiered. Ie A->B->C->Answer. As i jump from point to point to point without clear explanation.

Anyway, i'd be curious if my thinking would pose challenges for some sort of thought transcriber.

At some point, because machine learning is picking out algorithms from the training data, models will converge on functionally equivalent algorithms with the brain.

At first, it won't look good, because the training data will contain output from the millions of individual brains that produced it. Only the human processes most commonly shared will be learned, and the inference pass does one gigantic single run of a model to produce output. The brain has multiple parallel systems contributing to cognition, with hierarchical structures and cycles. If you could deconstruct transformer models into multiple networks, gradually mapping the architecture to that of a brain, performance should improve.

Even if you can't map at high resolution, every stage between monolithic and full cortical hierarchy should improve performance. At some point, such a system could be used in conjunction with Neuralink or direct bci to allow the brain direct access to a model - a true exocortex. With RETRO and lookup interfaces, you could have the whole of Wikipedia available to you at the speed of thought, or offload specific things like calculation and text memory storage.

Aside from the ethics involved, the speed of information consumption will have reached parity with the brain's native performance. Through training or natural development, offloading other functionality to the exocortex will improve on speed, with the biological brain learning to trigger exocortex processes. The neocortex would begin to operate more like an extended hippocampus, sitting at the highest level of the cognitive hierarchy.

This form of augmentation is explored in the Accelerando novel in depth, including the manipulation and temporary loss of the exocortex.

https://en.wikipedia.org/wiki/Accelerando

In 20 years, current gigantic language models should be trivially stored, executed, and trained on microsd sized chips. We'll have better bci, better batteries, better architecture than transformers, and hopefully a much better legal system around data privacy and surveillance.

If we avoid wireheading and obvious Black Mirror pitfalls, we could dodge the AI great filters that are lying in wait.

>hopefully a much better legal system around data privacy and surveillance If we avoid wireheading and obvious black mirror pits

Curious what you see as a possible incentive for this to play out? Personally stopped watching black mirror a few years ago, i dont know how much focusing the public psyche on a fantastical version of the horrors that are playing out helps us imagine or enable a better future. These days it almost feels like a keeper of the gate to STEM. If black mirror freaks you out you probably aren't going to compete in the r&d space.

The fall of Google and Facebook, Clearview AI and their ilk creeping in, like IRS facial recognition, and gradual adoption of sane privacy laws at the local and state levels. Boomers dying out and generation X on up will hopefully increase the number of technologically competent individuals in congress, too.

Right now, I see Google and Facebook as the only relevant fingers in the dykes. There are clear and grotesque violations of privacy happening that we simply don't have a proper framework for, and these companies are ruthlessly fighting to keep the profit flowing, despite the social costs.

I hope we can collectively get it right within 20 years, but access to the literal inner workings of a mind is a pretty simple example of private data everyone will want to be protected. Maybe. Hopefully.

> Imagine wearing a VR like headset that records fMRI/MEG signals and then instantly transcribes the words you’re thinking of into your text editor.

Now imagine being coerced into wearing this same setup except the transcription will be used for legal purposes. For years, each time we take another step towards something like this[0] I have sounded the alarm. It's like we techno-optimists never ever learn. The street finds its uses for tech, and so does Big Brother.

[0]: https://www.fastcompany.com/90350006/watch-this-device-trans...

For those interested, similar research has been underway since 2019 in the labs of Ev Fedorenko at MIT and Tom Mitchell at CMU.

Reply on news.ycombinator.com