Hacker News Reader

Comment by nemomarx | Hacker News Reader

I feel like I'd want the opposite of this - something that mimics the cleaning process of sleep that you could use in the afternoon for a nap or etc.

View on HN

Sleep isn't about cleaning. It might include cleaning, but I don't believe for a second that cleaning is the main function.

Why? Cleaning is easy to optimize yet sleep is evidently hard to optimize. Going unconscious for a big chunk of the day in a world where everything wants to eat you is a crazy price to pay, yet sleep is highly conserved across species under this extreme selective pressure. Even the animals famous for avoiding sleep don't really avoid sleep so much as bend the rules! Seals partially wake up to breathe, migrating birds sleep a hemisphere of their brain at a time, but they all sleep. This didn't happen because "pumps are slow" or "filtering is hard." Biology is excellent at copy-paste-reinventing that stuff. Optimizing the pumps and filters is not something evolution would have struggled to do if it could have meant not going unconscious for a third of the day. Especially for organisms with small brains where the transport logistics are trivial.

IMO the much more likely explanation is that the brain uses contrastive learning somewhere. We know animal brains don't use backpropagation to train and contrastive learning is one of the biggest competing family of techniques. Instead of using calculus and outer products to calculate derivatives, you feed real data and false data forward through the network and each neuron tracks and trains on the difference. This can be batched -- many CL techniques could handle 16 hours of real data followed by 8 hours of false data -- but the false data can't be put off forever, or the contrast signal goes away. Also, you probably want to inactivate I/O while the false data is flowing through. In other words: sleep emerges from first-principles on the category of neural network learning techniques that the brain probably uses.

I don't think anyone was seriously about to shoot up with sleep-replacement peptides, but also: don't. Let someone else figure this out the hard way.

I think you overestimate the disadvantages of sleep. It's very species dependent, for some it might be a disadvantage because you're more vulnerable to predators, for others it's an advantage because you spend less energy. If you're safe you really only need enough wake time to eat and reproduce.

Sleep has to be essential to life as we know it, because no animal has completely outevolved it. Some animals sleep an hemisphere at a time (dolphins, long-flight birds) or in fragments (giraffes), but none truly goes without sleeping no matter how challenging their environment is for a nap.

If you're safe.

You definitely see a bit of this in other primates. They sleep on trees (much safer) and for a lot more hours than humans.

That's great for primates, but what about nematodes? Everything with a nervous system sleeps, even when the cost is high.

Obviously sleep is necessary. I'm saying that there isn't always a large pressure to minimize sleep. Long sleep can be beneficial just to preserve energy.

Right, but my argument is "high pressure never managed to push past this barrier, therefore it's a strong barrier." The existence of other, lower pressure circumstances doesn't have any bearing on the conclusion -- unless you want to argue that the low pressure circumstance is so universal that the high pressure circumstance doesn't exist, that no animal anywhere is inconvenienced by the need to sleep. But I don't think that is what you were going for.

> Cleaning is easy to optimize

Your entire argument rests on this statement, yet I don't see any further talk about it or anything to back it up. Perhaps sleep is the very best way to optimise cleaning, outweighing the disadvantages?

Biology does an insane amount of packing and shipping in these spaces already and this is the stick against which we should measure potential modifications. While a neuron is active, vesicles will be riding up and down the axon tubules, shepherding RNA and building blocks, releasing and reuptaking neurotransmitters, and dozens of ion-selective transports create the electrical behavior. At the cellular level, the brain has its own immune system, at the tissue level it has the blood-brain barrier. This is all neuron-specific innovation, everything from basic cell biology still applies. The point is: if you are familiar with the biology, you are familiar with a dozen transport systems (and can infer the likely existence of a hundred mechanisms) more sophisticated than waste evacuation and none of them require the neuron to go offline, so the prospect that this is the hangup seems unlikely.

Also, the complexity of transport scales with the system. Famously, zebrafish don't need a functioning heart for the first week of their life because at that stage they are small enough that diffusion does a good enough job transporting nutrients and oxygen. Needless to say, this goes for waste evacuation as well: small size trivializes the problem. Nematodes are even smaller that zebrafish, yet they still sleep.

"Sleep is about training neuron functionality" explains the universality of sleep much better than "cleaning is hard."

If that were true, wouldn't you expect 50 year olds (who have mostly learned how to navigate life) to need little or no sleep? Seniors sleep only a little less than young adults, still 6-8 hours a night.

There are other known mechanisms in sleep, for example copying/condensing short-term to long-term memory by replaying it. Translating increased compression into better fundamental performance seems like a steep task and 50 year olds have already reproduced or not, so shrug

I think we made a big mistake by using biological terms for computer processes and now we have people like you coming to the """natural""" conclusion of going full circle and applying computer terms back to biological processes.

There is virtually no parrelel between both worlds besides the naming and no one has any idea about what's going on for real.

I don't see a need to be dismissive. There is a lot of evidence that there is a relationship betwwen sleep and learning. The hypothesis advanced by the OP is worth evaluating on its merits rather than being dismissed as a fale equivalence.

This smells a bit like a statement of faith. Digital neurons and neural networks are not the same as their biological counterparts, sure. But it is not an unreasonable proposition that their functions are isomorphic.

It is unreasonable, because a neuron is categorical different from a so called "neuron" used by AI.

Same is true when you compare an actual brain and an AI model.

> not an unreasonable

Yet we have absolutely 0 clue nor proof besides "it feels about right", that's not how the scientific method works.

It's not zero clue. Neurologists have got a lot of observations on how the brain works and there are multiple plausible proposed mechanisms for how the brain might do something like backpropagation, it's just that the details are really messy so there's not a single clear picture that has emerged.

(Similarly, there is decent evidence that the brain uses higher-dimensional vector representations of concepts in a similar way to neural nets, at least in some parts of it, but again the details are messy and there's not a clear idea of how these representations emerge or how to decode them reliably)

I keep hearing that. At the same time people makes comments like the one you just answered to, using that analogy heavily. Could someone who understands both say where the analogy breaks down in this case?

The brain is...

~a machine~

~a steam engine~

~a computer~

an LLM

Analogies help understand the brain but OP is taking the analogy and then going back and applying principles from the analog to the brain.

The brain is the brain, it isn't any of those other things.

While there is much to suggest that sleep is important for learning, why contrastive learning in particular? It seems to me that the "deactive IO" argument could apply to any learning technique. Similar to how railroad maintenance is carried out at night because it is too unsafe to do during the day when trains are whizzing by.

Contrastive learning provides an unusually strong push towards an offline solution, even though the offline solution is expensive.

Road maintenance isn't a good analogy; it _can_ be done "online" (with traffic, during the day) and in fact _is_ often done online, despite the pressure to do it online being less than "road workers will be eaten by monsters if they venture out between 8pm and 6am." This is the opposite circumstance we see with sleep. Animals always sleep despite the steep cost. If we want to fix the analogy, we could imagine a world in which all roadwork was done between 8pm and 6am despite workers regularly getting eaten by black-furred creatures of the night. Wouldn't that beg the question "why not do it during the day?" If you learned that every city and state and country did the same crazy overnight work schedules despite losing workers to them, wouldn't you suspect that there was a strong hidden reason?

Ok, but this has nothing to do with contrastive learning. Your argument applies to all kinds of self-supervised learning methods.

However, the contrastive learning hypothesis also explains dreams and why they make no sense.

I think it's often a bit of a mistake to assign one function to a given feature of a biological system. Biology tends to be messy and tangled and so something can quite easily have multiple functions.

Sleep is literally cleaning. It is flushing out the brain.

Cleaning happening during sleep doesn't mean it's the main purpose. It could be merely a convenient time for doing the majority of it.

dreams are false data

*synthetic training data

exactly!

> the much more likely explanation is that the brain uses contrastive learning somewhere

This doesn’t explain sleep in virtually brainless creatures. A metabolic root cause seems more likely.

Why would neurons be exempt from training just because they weren't clustered into a brain? By comparison, transport-related phenomena are almost completely trivialized in the simple creatures. The existence of sleep in simple critters seems strong evidence in favor of contrastive learning over cleaning explanations.

Could you make the case for metabolic explanations? Are there a bunch of coupled mechanisms that all require downtime and couldn't possibly be phased?

> Could you make the case for metabolic explanations?

Animals without nervous systems exhibit sleep-like states [1]. Potentially even animals without neurons [2].

[1] https://pubmed.ncbi.nlm.nih.gov/28943083/

[2] https://www.science.org/content/article/if-alive-sleeps-brai...

It's in the article no?

> Orexin antagonists (which block the molecules’ actions rather than mimicking it, and thus promote sleep), have been available since 2014 as treatments for insomnia. They work differently from conventional sleeping pills, which enhance GABA, one of the brain’s main inhibitory neurotransmitters. Birgitte Kornum, an expert on orexins at the University of Copenhagen, says such pills induce sedation rather than natural sleep and often lose effectiveness with continued use. Orexin antagonists instead turn down the brain’s wakefulness signal, letting its sleep-promoting systems take over. Evidence suggests this produces better sleep, with less risk of dependence and fewer long-term problems.

I’ve used both Belsomra (suvorexant) and Quviviq (daridorexant) and compared to the litany of other sleeping aids, Quviviq in particular is great for me in terms of not being groggy, not knocking me out standing up in case I somehow don’t sleep or need to do something. No rebound on days I run out or if I stop. Feels quite natural—I imagine it feels like what most normal sleepers feel like every evening. It’s nearly impossible for many to get them, tho…

Edit: To add some academic merit — there’s been research for its use for cognitive decline [0]

[0] https://www.nature.com/articles/s44323-025-00025-5

That's just a sleep med, right? I mean something to let you get the repair and cleaning faster, not just induce sleep.

If you want something that displaces adenosine from your neurons you are in luck, because it already exists and is very easy to get; It's called methamphetamine

Excellent, but now I have a new problem: An addiction to meth.

Is that worse than an addiction to sleep?

Do not, my friends, become addicted to sleep. It will take hold of you, and you will resent its absence!

Beware! If you start on a sleep addiction, one day you'll take a nap and never wake up!

Honestly that doesn't sound so bad.

What if we just normalized taking naps more often?

I feel like people insist that if you get 8 hours of good sleep you shouldn't need a nap

Not everyone can nap. Besides, the need to nap is an indicator of poor sleep. Why not just normalize sleeping properly (nb: rhetorical question)?

> the need to nap is an indicator of poor sleep.

Is it? Where do you get that from?

I remember the NASA doing extensive sleep studies, determining that the optimal pattern would consist of basically "naps only"?

I've been sleeping my adult life in usually 2x3-4h "shifts", which I also remember reading is a historically often used pattern. I do not see any negative effects.

Wild to read this about your sleep pattern. Please explain more. I find myself needing to sleep like 12+ hours every day.

https://en.wikipedia.org/wiki/Polyphasic_sleep

What do you want to know?

I do not track my sleeping hours. I guess it started in school, where I always stayed up way too late, so I used an afternoon sleep to catch up. I continued like that.

I despise alarm clocks, I think it goes against natural sleep rhythm and is bad for mental and physical health. We should ban them in civilized societies. I see it as one of the biggest luxuries of my life that I only have to use them on very limited occasions.

I also never grasped the concept of "every Monday at 10 am, you have to be willing and able to work". I don't understand how that can be done without messing you up after a few years.

I made it a necessity to only work jobs where I can decide on my hours with sufficient flexibility. Some weeks I work 80 hours if I feel it, some weeks I work only 10 if I don't. I co-founded a couple of projects, rarely had a boss to report to, and if I did, they cared about results and not when.

People are always more than happy with my 'productivity' and amazed at my flexibility. For clients/business partners, I actually seem to be available more often in terms of time slots than others with fixed "day" hours. I also mostly work(ed) in international teams, so people are all over the place in terms of timezone anyway (and a few at a time traveling around to conferences etc).

In my relationships, we either cuddle and I stay around until my partner sleeps and I quietly get up again to enjoy the quiet late hours (some of the best productivity, if I feel it!) and then come back later, or I sleep for a few hours next to them, and wake up early. I can then use early quiet hours for myself or for work, and then without stress prepare our breakfast. My second sleep shift is flexible, sometimes as early as 8 or 9 am, sometimes as late as 4pm - but usually within a certain more limited range for some months at a time. As long as I feel it.

I actually consider this to be "normal/natural", and what most other people seem to be doing "wild". :)

> I actually consider this to be "normal/natural", and what most other people seem to be doing "wild".

For example, I do not need stimulants like coffee or sugar drinks to get through my days.

By mimic do you mean it does the cleaning process or just that you feel like it did? If it was the former, that would be very interesting.

The latter is how caffeine works. Blocks adenosine receptors so you don't feel tired even though you are.

If you don't feel tired, are you really? I'm curious what are the objective metrics to determine someone is tired

Tiredness is a signal that the body/brain needs rest so the glymphatic system can go to work. You can lose the signal but still need rest. If your brain doesn't flush metabolic waste periodically you develop brain fog, bad memory, and increased risk of Alzheimer's, and you'll eventually die, as sleep deprivation studies have found.

Caffeine doesn't actually do any of the things sleep does to your brain. If you drink coffee during a late night, you won't feel as tired, but your brain performs worse, your reflexes are slower, you're less likely to remember things, etc. It's like painting over a rusted car instead of properly maintaining it, basically.

I think GP was just making a point about the fact that “tired” is a subjective experience, like pain. Arguably, if you’re on pain medication, you’re not in pain, even if your body is sending pain signals that your brain just isn’t receiving or paying attention to.

Indeed. Pain is also interesting because chronic pain, AFAIK is sometimes a misfiring of pain signals even when nothing is physically wrong.

Turning off that misfiring with drugs would be dangerous (pain is an alert) but also improve the quality of life of so many people!

Isn't the answer yes if people still die from overwork despite being caffeined up?

If I’m on morphine is my leg even still broken?

You have the ability to mentally push through a lot of stuff, but there are a host of signals your body will use to warn you such as: cognitive performance (although this is hard for yourself to actually admit its happening), behavioral indicators like yawning, head bobbing, frequent posture changes, and slowed speech are just a few indicators. Irritability is another big red flag.

I've only had two instances where I ignored all the signs and continue to push on until my body literally shut down on me. Finals week in college as an architecture student trying to finish a final studio project, and study for four finals. 5 days, 3 hours of sleep. That was one of them.

Michael Jackson is the ultimate case study for this.

so the former then

sorry I'm dumb and read it wrong

Reply on news.ycombinator.com