New theory proposes ‘forgetting’ is a form of learning | Hacker News Reader

New theory proposes ‘forgetting’ is a form of learning

tcd.ie

154 pointslaurex70 comments

My own experience: I play guitar, I like to think I play very well and lots of people have praised my playing. I do it in bursts, spending lots of time playing and practicing and recording for a stretch of some months then I… don’t play at all.

Life makes that cadence happen as much as anything, but all of my significant strides forward in technique and creative breakthrough have come shortly after breaking long dry spells. And that’s not to say there haven’t been disappointing breaks where everything feels cumbersome or fruitless. But all of my noticeable improvements playing have come after a significant stretch of time not playing, even if they were followed by significant time practicing.

I’ve been taking drawing classes and been trying to draw everyday for the last ~6 months, during the holidays I didn’t draw for 10 days due to being with family, when I got back to it it shocked me how much I had improved.

It’s so great when it happens! I’m glad you took a break and came back to big improvements.

Learning something like playing music, dancing, and many other things, is a process of becoming conscious of things, then learning to do the same things unconsciously (muscle memory), then becoming conscious of how you're not doing them right and correcting that, then again going back to doing those things unconsciously, lather, rinse, repeat until expert, then do it again.

All true but (again in my experience), I’ve found muscle memory improvements by basically dropping all of my work on developing it and staring all over, and I’ve found music writing/stylistic creativity improvements by just atrophying muscle memory and coming back feeling like I don’t know anything [about what I’m trying to do] at all.

I have experienced something similar in my general learning patterns -- and, being 35+, I don't think this has always been like that. Might well be that information overload etc has actually amplified this kind of impulsive learning sprints ("learning in bursts") for many of us.

This definitely shifted for me with age, with (again for me I don’t want to overgeneralize) bigger improvements and bigger gaps as I aged. I don’t think I could seriously describe playing guitar well until I was in my late 20s and couldn’t say it with much confidence until my mid 30s.

As far as information overload, it was my late-mid 30s when I was diagnosed with ADHD, and I have a lot of sensory sensitivities. I barely even listen to music right now.

I have experienced exactly what you're describing. I want to find out what the minimum effective layoff time is, however, so I can do it deliberately rather than randomly, and hopefully maximize the rate at which my playing progresses.

I don’t know if it works that way. I vouched for your comment because I think it’s a bummer it was dead, but I can’t say I’ve ever used breaks or falling out of practice as leverage and I don’t think it would be healthy for me to think of it like that.

Enjoy taking a break. It’s good for you, or at least necessary when you need to. Be present and grateful when you come back from a break and see improvements. And build on that if you feel like you have the momentum then.

I have seen this too. My pet theory is that, at all times, the brain is busy doing gradient descent on all the problems you have ever encountered in your life. Everything is getting improved in parallel. So when you step away from the problem for months at a time and come back you are reaping the benefits of all that background computation.

Alternatively, you where doing gradient decent with daily practice but got stuck at a local minima. Taking a real break helps you get past that.

However, I don’t think either represent what’s actually happening very well.

I think what happens is that during the active learning part you heavily overfit and end up in a local minima. Afterwards life acts like a regularisation, and when you return to it you are no longer in a minima but now have a gradient to descend on again.

I've experienced the same thing, but always had to wonder: maybe you're on an underlying trajectory of improvement fueled by practice, which is not influenced by brief breaks.

However, after a break, you notice all the improvement that has accumulated during the break. By contrast, when you practice daily, the day to day changes are small and you don't notice them.

I can relate, and I have to commend how well your comment put my in the picture for the article.

I have another hobby, that is language learning and etymology, which is notoriously plagued by false cognates and false friends. Arguably, forgetting about illconceived notions requires inhibition. Once I forgot about it, I will gladly accept the mainstream opinion.

I wonder if this is related to the effect of being better at something after not doing it for a while.

When it comes to a lot of things I do repetitively, I notice that there comes a point where either my skill doesn't improve or I actually get worse at what I'm doing. Then I'll take a day or more off, and when I come back suddenly I'm a total boss. The brain probably needs rest time to throw out the bad input and that can't happen as much during execution.

An old mentor of mine once said some thing I found very profound: “Learning is like eating; you can choose what you consume. But knowing is like digesting; it will happen regardless, and in its own time. You only have control over eating, so eat well and frequently”

There’s really a little leap of faith any time we want to learn something, because we don’t actually control how or when it fits into our brain or body, just that if we keep trying that it will.

I frequently find myself contributing to conversations with knowledge that I don't recall specifically learning. Once in a while, I think "Wow, how did I come up with something that insightful? When did I ever learn that?"

It seems there's a background activity in the mind that is actually synthesizing a wide variety of inputs from our environment. As in what I read, movies I watch, meetings I sit in on, Tweets that transpire, hikes I take, visits to distant cities, and so on. In many cases, it seems the connections and support for ideas, notions and realizations that then become knowledge are made without explicitly reasoning through them.

I'd say this happens most frequently in conversations related to my work, which makes sense since that's where most of the inputs are sourced.

> Once in a while, I think "Wow, how did I come up with something that insightful? When did I ever learn that?"

... and then I look it up and usually realize I was way off :)

innsæi

means "intuition" in ancient icelandic

I think this is what happens during sleep (a part of it).

There have been times where I've been unable to "fall" asleep, but stayed in my bed, completely unaware of my surroundings or anything else except what's going on inside my mind.

It feels like my thoughts were starting from a single point, then randomly branching off into multiple directions (one after the brisk "completion" of the other), and each path had a certain "feel" to it -- as if my brain was testing things it had "learned" or "inferred" (and the mental models it had collected), or even things it wasn't too certain about or even guessing -- simply to see how it would "feel" when thought.

It's like your mind has a complex array of filters/gates that decide how an input will be processed -- and these filters are constantly changing, rearranging, and growing more complex as you grow; so during sleep it feels like the mind is in a loop, constantly throwing very simple things that have collected in memory, into the "mix"/filters to see what happens (will those filters adjust, a la conditioning and adjusting weights like in a neural net? Will some new and palpable way of looking at things be found? Are some thoughts/paths no longer needed? Should other thoughts/paths be prioritized?).

I think our minds hold onto all the inputs we've gotten during the day, and then process them at night, is what I'm trying to say. I imagine this is one of the big reasons why we sleep so much as we do (compared to hunger-gatherers, and primitive tribes): we have so much sensory input that gets collected during the day (including our thoughts, albeit the thinking we do during the day serves the same end: forcing our heads to deal with new information to figure out what to do with it).

And then you have meditation, that basically forces you to clean out the "volatile memory," so it doesn't interefere throughout the day. Basically telling your mind, "I don't need to incorporate this shit into my thinking. It's not vital. Dump the memory onto the disk, and we'll deal with it later, but not now!"

I've noticed that sleep completely changes my train of thought.

On any given day, I'll have a certain "mindset" or overarching "feeling" for the day. If I stay up for a day or two, that feeling will remain. And no matter what I do, that feeling rarely changes throughout the day. BUT, when I go to sleep, it's completely different. I always wake up with a new "train" of thought. Like when I was awake, the train was going on a certain rail line, but during sleep it was moved to a different one.

Related to this: Since when I was young, maybe around six or so, on nights before special days I looked forward to a lot (like my birthday), I was desperate and agonized over the realization that my current self (what you call "train of thought") would end once I fell asleep, so this current longing that I had would never actually be fulfilled. This made me really sad, and it felt like my current "self" would die and get replaced the next morning by a somehow related "self", that was in it's essential emotional state very new and in that regard definitely separate. I continued to have this feeling from time to time (maybe 2-3 times per year, less later on) until well into my twenties, when it somehow receded. I have not talked about this often, but when I did, I never found anybody that could relate.

You may find this comic speaks to you quite a bit:

https://existentialcomics.com/comic/1

I've discovered much the same. If you visualize yourself doing the activity during the break, it actually accelerates the learning process.

I began playing piano last year and this blew my mind. I’d play 30-60 mins a day, and every single time this was true: I was better on my first try the next day than my best try the previous day.

I would say this has to do with sleeping. You've realigned everything and the signals come clearer the next time around.

I wonder at what stage that takes place. This seems to happen regardless of what takes place in dreams, though I know that dreams take place only in certain stages.

That’s the theory behind Spaced Repetition Learning, and it doesn’t surprise me that it applies to physical skills.

And people with PTSD and anxiety have problem with sorting what to forget. It may be just that troubled mind cannot forget because it thinks that this one bad thing is important, even when in grand scheme of things it isn’t. Maybe that is why psychedelics help people with ptsd, it rewires the importance of memories. Just my drunken 2c.

That's great. I've always thought forgetting was a key part of learning. When you are learning something, you pick up a lot of bad habits. Part of learning is forgetting those bad habits so they don't override the good habits. This is why you want to start slowly and work up to a faster speed, so you can make sure you're practicing good habits. If you try to go fast at the start, you will be going too fast to control what you're doing and will drill bad habits into your muscle memory instead. It can be really hard to get rid of those bad habits after they've cemented.

Exactly, I have noticed this same pattern and came to the same conclusions, but not only are you forgetting the 'bad habits' you're also integrating micro habits/skills into more general ones with this same process. In short: adding noise to noise/signal ratio can enhance the type and strength of signal.

Yes. An old weightlifting coach I had would say this. Every time you practice a lift you do some of the “right” things every time. And some of the “wrong” things too. But there are many more ways to be wrong than right so the right things start to reinforce. And then you take a break and your body starts to forget the movements. But the diffusely learned wrong things dissipate faster so you’re left with more right than wrong. Over many cycles of this you learn the right way to perform the movements.

My ex is a linguist, and she used to tell me about the “learning by forgetting” theory of the “phonological inventory” — which is a fancy way of saying “the sounds you can distinguish and produce.” The idea was that children were born able to distinguish all possible sounds, and over time forget the ones they are not exposed to. Interestingly she was a syntactician by training and would tell me that there are many people who believe this is true for syntax too: that children are born with all possible grammars, and “forget” the parts that they’re not exposed to. So if a language expresses parameter X but the child hears ~X, the child “forgets” about X and learns ~X by forgetting. This kind of supposes that a grammar is a set of discrete rules. But I’m not an expert in any of this, for details you’ll need to track down a real linguist.

Interesting. Is there a correspondence between when a child loses their superhuman ability to learn new languages, and when a child achieves proficiency in the grammar and phonology of their native language?

Your comment, along with the article, confirm my biased beliefs that humans are less pattern matching machines than noise filtering machines. Pattern recognition is less about connecting dots than it is distinguishing and purging the unconnected. When that process goes awry, and we can’t discern disconnections, things like psychosis arise.

Tangentially, it stimulates a connection to another though that sometimes flares up concerning evolution and specialization.

> If memories were gained in circumstances that are not wholly relevant to the current environment, forgetting them can be a positive change that improves our wellbeing.

If as a species we evolved and specialized for a certain environment, and now we have changed the environment so much that those specializations are a hindrance, would a past proto human be better at adapting to today’s world, if brought up by modern society, due to better adaptability? Biodiversity is the blank slate beginning, and selection is the forgetting.

What also does this mean for psychedelics? Could they "reopen" the filters that were earlier preventing new signals from being found?

Something like switching from a digital radio dial (every .2 MHz) to an analog dial (everything in between) - you can find something you had missed before.

It does make you wonder if we “learn” that things are NOT connected, and can chemically stimulate reevaluation of such learned non-patterns, helping break preconceived notions.

Like if after being taught oil and water don’t mix, you could unlock the ability to question such belief.

> Could they "reopen" the filters that were earlier preventing new signals from being found?

This actually is a leading theory of psychedelic action, ie. that they tear down filters we use to operate in the world.

> Pattern recognition is less about connecting dots than it is distinguishing and purging the unconnected. When that process goes awry, and we can’t discern disconnections, things like psychosis arise.

Making connections that aren't there, ie failing to purge the unconnected, could be psychosis. The opposite would be: Not making connections, ie purging the connected. That is called low intelligence.

High intelligence is often seen as making connections. Do intelligence and psychosis correlate? (And if you can make someone purge connections, are you lowering their intelligence?)

I don't know how much of this has to do with forgetting so much as attaching significance. One thing I realized I was able to do for much of my life (perhaps not quite as well as I used to) is keep the concept of sound and word separate. Of course we can do this for sound effects. I've been able to maintain this for most word sounds. This was very apparent when learning foreign languages, reproducing the sounds was the easy part--just make the noise you hear.

That is synaptic pruning for language.

https://en.m.wikipedia.org/wiki/Synaptic_pruning

I forget everything I didn't figure out myself. Some books I struggled, looked at the answers, and I can't remember anything I did 5+ years later. Other books/problems I struggled, figured it out myself, and I can recreate that solution like it was yesterday.

This reminds me of a funny passage in Jin Yong’s wuxia novel “The Heaven Sword and Dragon Saber”, where Zhang Wuji, the protagonist, learns taichi sword from his grand master Zhang Sanfeng.

Zhang Sanfeng demonstrated the whole taichi sword form to Zhang Wuji repeatedly, and after each demonstration, asked Zhang Wuji “how much have you forgotten?”

Zhang Wuji instantly got it and each time replied that he has forgotten more and more, while Zhang Wuji’s subordinates watched on the side and became more and more worried that their hierarch was forgetting his martial arts.

> Jin [...] wuxia [...] Zhang Wuji [...] taichi [...] Zhang [...].

One of these things is not like the others. Why spell "taichi" that way?

I was typing from a phone and didn't pay attention.

Why do you feel the need to point out a tiny error when a Chinese person is going to the effort of romanising their own language so that we can all read it?

It's unlikely to be an error; no one is going to confuse ji with chi. I wanted to know the thought process. I was expecting something along the lines of "I was using 'taichi' as an English word, not a Chinese word".

Well, when you put it that way, it's pretty much like you said: "taichi" as an English word. I've always known "taichi" instead of "taiji" in English, even though I know the correct pinyin.

Another reason I think forgetting is so important to continuous learning is that if new ideas conflict with accepted ideas, it will be more difficult to integrate them. Like adding to a map or puzzle where the boundaries don't mesh. So by having fewer deeply held beliefs, new ideas are free to enter and coexist. You can also compartmentalize to some extent like superpositions of possibilities where each one can have new consistent information added without regard to how it conflicts with other possibilities. Not that I have a clue if this is a good example (or advisable) but a physicist could learn both loop quantum gravity and string theory and relate them both to understanding our world and keeping them separate could be helpful for learning.

I read something interesting recently about how most discussion of brain function is 'tainted' by metaphor, these days usually involving computers. Basically the concept was that our brains don't process and store information like a computer does, but because computers come the closest to accomplishing what our brains do it's the easiest point of reference for us when discussing brain function.

Since I don't really know anything about brain science and know a little about computer science I find it pretty hard to escape that metaphor and imagine how a brain can function without doing what a computer does. This type of article does give me a bit of an inkling though.

I wonder if conversely computer architecture is similar to the brain in at least some macro-structural ways because something about the function of the brain makes it easier for it to design something similar to itself.

look into neuromorphic computing

Isn't this similar to dropout in machine learning neural networks? [0]

https://en.wikipedia.org/wiki/Dilution_(neural_networks)

Dropout is random, rather than learned. But I can see how it could have a similar effect to forgetting.

I was thinking something along those lines too. As if a baby starts with a fully connected network of many nodes, then over time experience strengthens some connections as others drop out. If the connections were to return to an evenly connected state everything would have been forgotten, but anything could then be learned as all capacity to learn has returned.

would dropout be considered forgetting or learning to use as little information needed to get the right answer?

Forgetting is similar to regularization mechanisms in deep learning like dropout. You tend to forget details, and this makes your mental models simpler over time. The memories that get refreshed are the details that actually matter and keep tripping you up

I have always been forgetful. I have never thought that was a problem. I often experience same things with repeated “love in first sight” reaction. My friends keep reminding we have done this before and I would not have recollection of that. I seem to have intentionally forgetting certain things just to experience them again, or not overthinking them that would cause any anxiety.

That’s why I hate year long lawsuits. I keep have to remember the details until the cases are settled. I want to move into woods to live in peace…

“the engram – the physical embodiment of a memory”

This sounds like a grotesquely inadequate explanation of memory, along the lines of "vapours"

“My brain is only a receiver, in the Universe there is a core from which we obtain knowledge, strength and inspiration. I have not penetrated into the secrets of this core, but I know that it exists.” ~ Nikola Tesla

I rather like this idea that my brain is a receiver and my memories are stored in the cloud

My piano teacher always told me "you learn to skate in the Summer and swim in the Winter" according to the Russians.

I find my forgetting pattern similar to a garbage collector — not sure how universal it is, but I tend to forget things I no longer have connections to. Eg. I will quickly forget memories with people I no longer communicate.

I'm reasonably certain that I read very similar things in some writings of Piotr Woźniak on the things he found out while working on SuperMemo. Must have been at least ten years ago.

This is a blindingly obvious hypothesis. Assuming the mind has finite hierarchical memory, this is just a matter of purging cache or overwriting other information.

I’m learning so much, including my name every day! Wait, what were we talking about?

Is this really anew theory?

that's what i was thinking too. we've known this from neuroscience for a while. i at least remember learning this from at least 5 or more years ago. the general idea being that forgetting is an evolutionary advantage to help consume less conscious memory/processing. but there's always the chance that i'm misinterpreting this particular finding.

edit: was going to add another comment, but i've forgotten it now. something about how this is related to what neuroplasticity is. the brain getting rid of connections that aren't being used

The irony if this was a forgotten theory.

Good news for me.

The brain does garbage collection when you sleep. Probably it does an even wider reorganization when you leave some neural circuits unused for some time.

Personally, with gaming, I had some improvements coming from restarting a habit but psychologically it is harder to suck more at something and working to get to your former performance. I have seen it in a lot of gamers. They get gud, for some reason they grow tired and leave, they return, they wonder at how out of shape they are, they play a while and leave again when they realize how much they have to work to get to top tier. Ofc I am not referring to games based on the amount of time and money you waste on them, but skill based old school FPS shooters.

The heaviest alcohol drinkers in my society are university students. Seems like a correlation.

May be important to note that "hyper indoctrination" vs. memorizing too much at once leads to rigidity or inefficiency and recall - and so breaking down those walls a bit may allow info to be more easily accessed; e.g. alcohol makes someone "looser" - their stable pathways for memorization that have been developed over an open mind (neural network) vs. critical thinking development [where there aren't lists to memorize but you're exposed instead to concepts, patterns, and frameworks - and naturally you remember automatically because there's enough interest when you're reading it (and strength of importance derived from the underlying interconnectivity/passion, vs. forcing yourself to memorize because it's on a test).

Reply on news.ycombinator.com