The brain simulates actions and their consequences during REM sleep | Hacker News Reader

The brain simulates actions and their consequences during REM sleep

biorxiv.org

196 pointsXzetaU863 comments

I've often thought this was the case. Tangentially, when I don't get enough sleep, and my executive function is impaired, I tend to drift into involuntary daydreams. The content of those daydreams trends to feel like simulations of future events and possible outcomes. I can't say it's related, but I often wondered if my brain is saying "If your not going to sleep enough, so be it, I'll try to squeeze in some simulation tasks during the day". When I was a teen, I'd daydream about skateboarding. It could feel so realistic that I could get a "jump scare" in imagined situations that ended in me hitting the ground.

You’ve succinctly described what happens to me when I don’t sleep too well, even down the sudden jump scare when something happens in the day dream

You might be microsleeping

https://www.cdc.gov/niosh/work-hour-training-for-nurses/long....

“The human brain can respond to sleep deprivation by reducing alertness and generating microsleeps – involuntary episodes of sleep lasting a few seconds.

During microsleep, you may appear to be awake (eyes open), but your brain will not process information. Thus, lapses in attention occur.“

I seem to have microsleep trigger when my wife is telling me we've got something planned in three months' time.

In three months minus one day, she'll let me know (again) and, damn if it's not the first I'd heard of that, and damn if my reaction doesn't cause an argument.

it's really weird when they start conversations by saying "you weren't listening to me, were you?"

I wouldn't be surprised if you told that ADHD has been diagnosed.

If that's happening all the time as GP says, probably true.

I do like reminding people that a lot of strategies / techniques / process improvements which help people with ADHD, also really help non-ADHD people as well. In this case, even someone without ADHD can occasionally forget or not mentally "record" a specific event/calendar date. Both ADHD and non-ADHD people would benefit from a calendar maintained by both them and their partner, along with both partners reviewing the calendar together. And periodic verbal reminders leading up to the day rather than just the day the event is agreed to and the day before the scheduled event.

The goal of most ADHD time-management techniques is to offload what you can out of your executive functioning brain / working memory, so that you can use as much of your working memory as possible for other unplanned & dynamic things that pop up throughout the day. Even if you have an excellent working memory, you still benefit from offloading stuff that doesn't really need to be there.

We recently (-ish) started using a shared calendar.

Issues only crop up for things not put into it, or when they don't have a good breadcrumb trail of reminders leading up to it.

The main issue with my lack of remembering / hearing / listening / caring is that as soon as I walk in the door, before my bag is put down, before I've had time to change out of my work clothes and maybe take a breath, I'm faced with the onslaught of a detailed, step by step account of her day, interspersed with "by the way, we're going to X with Y on the Zth of September". There's no algorithm to separate the small amount of wheat from the abundance of chaff.

I'm a complete arse, but she only notices when I say "you didn't tell me about that!?".

I recognize myself in your description. So probably I need to diagnose myself? /s

I've long bad a hypothesis that dreams are kinda like a training simulator where you can test your reactions. This is then esp. useful for extreme situations (simulated as nightmares) - like you can't really learn in the real world how to react when you encounter a grizzly bear for example - you can't mess it up the first time or you're dead. But if you live in Alaska, you're likely to have such a dream and have the opportunity to try out such a scenario.

Anecdotally: I live in bear country but never dreamt about bear encounters. I dreamt a lot about wolves as a kid but only twice after i moved to a place where wolves are, and in such unrealistic way that I'm fairly sure it would be moot as practice/hypothesis testing. Most of my learnings in dealing with wildlife come from things like bear aware training and real life encounters.

It's hard to state 'I've never dreamt about X', its more appropriate to state 'I've never remembered a dream about X'.

Fair enough. I remember my dreams most days, often multiple per night, but you're right that I can't be sure I remember all of them.

Have you tried a sleep study? My life has been spiraling down the tubes until recently. Sleep apnea will wreck your life.

Not yet, but that's not a bad idea.

I thought this was already understood? Are anyone working on an analogous way of integrating reinforcement learning into training generative models based on what they have experienced in user interactions? Seems like having models “dream” e.g. do hypothesis testing and correct the weights based on the results could be a way out of the lack of enough training data.

The mainstream info is predominantly that dreams are basically nonsense content (e.g. people dream they can fly) when executive functions are turned off and brain is doing garbage collection and memory consolidation, whereas this title argues the brain is nevertheless learning causal information from dreams which would suggest a more active role than previously thought.

I'm with the GP, I thought I had heard this somewhere before. Also, I don't think both propositions are in conflict. Dreams may be meaningless and the brain may be using them for something.

My hunch (based on my personal experiences) is that these occur in different phases of sleep.

I find that while I am falling asleep, I often have indescribable nonsense "hallucinations" (I hesitate to call them that because it makes them sound a lot more cohesive than they really are) that feel almost like random noise. These are not just sensory hallucinations, but also logical hallucinations: I have thoughts and beliefs that make so little rational sense that I don't know how to describe them.

But later on I will have complex dreams with coherent characters, plots, etc., and have an inner sense of logic (even though it is dream logic) that I can describe after I wake up.

With regard to your “hallucinations” comment, you remind me of something the old hippies told me. I remember asking about acid flashbacks and the heavy users would say oh yes we get flashbacks and I’d ask them lots of questions, eventually I established with them that the most likely time to experience as flashback is falling asleep or waking up from an afternoon nap.

On a very rare occasion my brain gets stuck for a few minutes in the state between sleep and consciousness and this kinda does feel like acid.

Hallucinations and lucid dreams during falling asleep are called hypnagogia and definitely are different as they don’t involve REM. I think a lot of people don’t talk about it for fear of sounding crazy.

https://en.m.wikipedia.org/wiki/Hypnagogia

And for those interested In hallucinations, Oliver Sacks has an amazing book that I think also destigmatizes them and covers how different they can be from hypnagogic, to migraine, to drug induced, etc.

https://en.m.wikipedia.org/wiki/Hallucinations_(book)

I think in RL engineering circles offline, off-policy, generative experience replay is. In neuroscience, the idea is being actively explored, but it isn't a consensus position.

I've had plenty of dreams where I woke up thinking "I definitely won't do that."

Definitely. I think a lot of deja vu and deja reve moments are actually these simulations happening at night.

I believe this paper shows that a part of the mouse brain responsible for telling muscles how to move when orienting also correlates with direction specific firings of neurons that are likely responsible for tracking the animal’s orientation and this occurs both while the mouse is awake and asleep. This updating of heading while asleep is considered a simulation. They take steps to experimentally show that this relationship exists and is likely causal.

This paper doesn’t show that simulation is the purpose of REM (though it may be) however they do go on to speculate that the simulation that is occurring could be used to improve the brain’s predictive models of movement actions and their outcomes without needing to move the muscles.

It is thus conceivable that the internal model that coordinates activity between SC and ADN observed during REM sleep may be at work also in the awake animal. This would allow the brain to predict shifts in head direction following the issuing of a motor command in the SC, before the motor command is executed and thus have a real time estimate of heading rather than depending on the necessarily delayed sensory feedback from the vestibular canals. … In the absence of error signals, these internal models are free to unfold and may represent the building blocks of the brain’s generative activity during REM or REM-like sleep in humans and animals.

This would explain why the answers to some of the hardest issues i've worked on seem to come to me after I wake up.

Hammock-driven development is a thing: https://www.youtube.com/watch?v=f84n5oFoZBc

I think anyone can experience this even if you're unaware of the science behind it. The brain during sleep can also connect seemingly disconnected things in ways that your daily logical process would have a lot of trouble with. That's why things seem to "click" when you wake up from sleep. If your daily thinking is a gradient descent, your sleep thinking is a flood from all points and in all directions. But I don't have any science refs on this; just my experience.

In my last role I was working remotely, and almost every day after lunch I'd have a nap on the sofa for 20 minutes with the cat.

In that role I felt the most productive I've ever been, and I suspect it is this habit that made it so. During this time I'd often come up with solutions or new ideas to problems.

Now I work in office, and even if it was socially acceptable to nap after lunch, there is no sofa or cat.

Sleep is quantum problem solving mode?

While I have complex dreams, they seem completely unrelated to anything that I was thinking about during the previous day, in any case completely unrelated to any professional problems.

Despite that, it is extremely frequent that in the very instant when I become awake in the morning I am aware of a solution to whatever problem I could not solve the previous day, so it is very common for me to start a day by jumping from the bed and going immediately to a computer, to turn it on and write down the new solution before I forget any details.

I wonder if this can apply to humans for situations that either resolve conflict or to anticipate future events more effectively? People are known to process traumatic memories from the past during REM sleep by essentially reliving the experience and they're known to work through potential future events e.g. business executives have simulated meetings in their dreams for ones they anticipate to happen soon in the future.

Makes sense to me. I organise a lot of hiking trips and I frequently have dreams where one of my trips has gone wrong and I'm trying to cope. It's a cheap way to simulate a problematic situation.

This reminds me of an experiment in grad school on spatial memory in rats. We noticed that during sleep their hippocampal place cells would fire in a way similar to those seen during maze navigation. I personally experience this all the time, if I've been programming all day, I will be in a 'dream-like' state within the program

The infamous Tetris brain. I don't feel it helps my sleep though. Usually im just solving problems that don't exist anymore.

Yes - here's a short TED talk on the topic:

https://youtu.be/Vf_m65MLdLI?feature=shared

Only read the abstract, but I wonder if some consequence to this field of research leads to our ability to control movement in dreams using some kind of external rf. The "control" might not be crisp without a visual component, but even having some ability to issue commands would be incredible.

I hope that nothing external can control my dreams.

You can control your own dreams by practicing lucid dreaming.

We know the end-game of such technology would be to serve ads to you while you're sleeping.

How do we know ads do not influence our dreams already?

Maybe they do, but at least Google isn't charging for dream impressions yet.

They do, Coca Cola been in my dreams before as product placement.

https://x.com/propheticai/status/1803811493881798701 & http://bit.ly/3WumKvu

  Prophetic is developing a non-invasive neuromodulatory wearable. The device uses various “reading” neurotechnologies— including EEG and fNIRS, as well as a “writing” neurotechnology— transcranial focused ultrasound (tFUS). Our aim is to develop a viable neuromodulatory brain-computer interface for consumer use that is 1) closed-loop, 2) non-invasive, and 3) possesses a temporospatial resolution meaningful enough to induce discrete states of consciousness—such as lucid dreaming—with precision.

Just needs something about longevity or crypto and that’s the trifecta of bs if Ive ever seen it. Yes im pessimistic, no offense to the company or researchers.

If they publish the results of their VC-funded research, the scientific community can benefit.

I've had so many dreams where it seemed like my brain was simulating some unusual social situation so that I would know how to behave and react in such a situation in real life. Especially situations that may cause stress or anxiety.

So… it’s training itself on synthetic data?

Thats what imagination is.

Then it must be really efficient amount of data since we only get like 7 dreams a night

the biological grok

running inference

So: "Sleep on it" is an effective strategy.

It reminds me of some talk I had the chance to come across, I can’t remember which one. Sleep could be a training phase. Discerning whether inputs come from the real world or from the trained neural network itself would be the reward function. When it’s no longer possible to discerne reality from "fiction", the training is complete. And this could be somehow linked to the ability of training while simultaneously operating the network. I wish I remembered where I heard about it.

Edit: I think this was it: https://www.youtube.com/watch?v=FBkhbqrFyo4

Why does my brain have to simulate so much of weird and stressful bullshit adventures which can never happen IRL? If it mostly is concerned about how to move the limbs then why I never remember actually using my body to do anything in a dream except the sensation I invoke to fly? To me it feels like my dreaming brain is mostly simulating emotional experiences.

During sleep the brain might be constantly simulating past experiences, movement, and altered consequences as a possible "re-training" model. Some of those simulations may have outsized emotional moments which "jolt" our consciousness enough to perceive the dream. Painful memories are often an easy target for these simulations to use because they have more definition that many other memories.

These unrealistic adventures may not be for training you how to deal with unrealistic adventures. These dreams could be serving a much lower-level purpose: training your brain's deeper inner workings that you do not have direct conscious access to. Flying may help with fine tuning visual-spatial processing, for example.

yeah, it's never really flying for me, but lots of navigating across fantasy landscapes in odd modes of transport like chairlifts and ski runs. But the perception of those landscapes is very much including bird's eye view, only that it's never "me" at the elevated camera position. Whatever story happens, it simply isn't told exclusively through "subjective camera". (does dreaming even involve first person view? I think it's more like a faux awareness of being somewhere than faux vision)

It's not flying to me, more like being able to "feel a map projection". But that's certainly close to flying, I guess it's the same category of dream from a "why" standpoint. And once the "i can fly!" part of off the table, I think the "why" is rather obvious: get used to thinking in larger topologies than what we encounter in sedentary day to day life.

off-policy, offline, generative experience replay: these are all efficient ways to maximize our learning with finite experiences to learn from. for example, its hard to learn from rare experiences, and dangerous to learn from very hazardous situations you'd not want to be in.

However, as you pointed out, generative experience replay runs the risk of learning from counterfactual or impossible experiences. To mitigate this risk, we need a discriminator network, not only to distinguish dream from reality but to judge their plausibility (e.g. flying by flapping wings vs dreaming you took a vacation in Tahiti that you never took vs dreaming about forgetting to do your homework).

How prevalent is REM sleep? Does it map to sentience?

(Fun aside: land mammals are the weirdos of the animal kingdom in that we turn our whole brain “off” at once when we sleep. Most animals rest part of their brain at a time.)

> This study suggests that the sleeping brain, while disengaged from the external world, uses its internal model of the world to simulate interactions with it.

So we do live in a simulation, for a few hours each day.

I wonder if this is why we generally experience making better decisions when we have habits that foster good quality sleep.

Looks interesting. When will the article be peer reviewed ?

The purpose of sleep is to prepare a species for battle

Reply on news.ycombinator.com